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Abstract 


Force  development  activities  play  a  major  role  in  ensuring  that  the  Canadian  Forces  evolve  and 
adapt  to  meet  future  mission  requirements.  However,  the  various  joint  force  development 
activities  (capability-based  planning,  concept  development  and  experimentation)  are  not  closely 
integrated  and  frequent  changes  to  these  activities  have  been  implemented  in  recent  years.  With 
the  intent  of  guiding  potential  future  changes,  this  report  provides  best  practices  on  organization 
development  based  on  a  review  of  the  scientific  literature.  An  assessment  of  the  current  joint 
force  development  activities  and  their  limitations  is  then  performed.  Finally,  the  paper  provides 
recommendations  with  regards  to  Concept  Development  and  Experimentation  and  how  these 
activities  can  best  contribute  to  the  force  development  objectives. 

The  report  highlights  the  various  aspects  that  should  be  part  of  a  systemic  force  development 
activity.  It  is  found  that  some  of  these  aspects  are  currently  ignored  within  the  current  capability- 
based  planning  approach.  These  aspects  are  also  used  to  recommend  specific  human  views  that 
should  be  included  within  the  Department  of  National  Defence  Architecture  Framework.  The 
report  also  highlights  that  concept  development  should  deliver  well  defined  and  precise  concepts 
for  which  all  potential  trade-offs  have  been  clearly  identified. 


Resume 


Les  activites  de  developpement  des  forces  jouent  un  role  important  lorsqu’il  s’agit  de  faire 
evoluer  les  Forces  canadiennes  de  maniere  a  ce  qu’elles  s’adaptent  aux  exigences  des  missions 
futures.  Toutefois,  les  diverses  activites  interarmees  de  developpement  des  forces  (planification 
axee  sur  les  capacites,  elaboration  et  experimentation  de  concepts)  ne  sont  pas  bien  integrees  et 
ont  subi  de  nombreux  changements  au  cours  des  demieres  annees.  Dans  le  but  d’orienter  les 
prochains  changements  qui  pourraient  etre  apportes,  le  present  rapport  s’inspire  de  l’examen  de  la 
documentation  scientifique  pour  presenter  les  meilleurs  pratiques  en  matiere  de  developpement 
organisationnel.  S’ensuit  une  evaluation  des  activites  interarmees  de  developpement  des  forces  et 
de  leurs  limites.  L’etude  se  conclut  par  des  recommandations  relatives  a  l’elaboration  et 
1’ experimentation  de  concepts  et  concemant  la  maniere  dont  ces  activites  pourraient  le  mieux 
contribuer  a  Tatteinte  des  objectifs  de  developpement  des  forces. 

Le  rapport  fait  ressortir  les  divers  aspects  qui  devraient  etre  pris  en  compte  dans  une  activite 
systemique  de  developpement  des  forces.  On  constate  que  certains  de  ces  aspects  ne  sont 
actuellement  pas  pris  en  compte  dans  le  cadre  de  l’approche  actuelle  de  planification  axee  sur  les 
capacites.  On  se  fonde  egalement  sur  ces  aspects  pour  recommander  certains  points  de  vue 
humains  qui  devraient  etre  inclus  dans  le  Cadre  d’ architecture  du  ministere  de  la  Defense 
nationale.  Le  rapport  souligne  que  l’elaboration  de  concepts  devrait  donner  lieu  a  la  creation  de 
concepts  bien  definis  et  precis  pour  lesquels  tous  les  compromis  potentiels  ont  ete  recenses  avec 
clarte. 
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Executive  summary 


Joint  Concept  Development  and  Experimentation:  A  Force 
Development  Perspective 

Dave  Allen;  DRDC  CORA  TM  2012-036;  Defence  R&D  Canada  -  CORA; 

February  2012. 

Background:  Force  development  activities  play  a  major  role  in  ensuring  that  the  Canadian 
Forces  evolve  and  adapt  to  meet  future  mission  requirements.  Due  to  the  evolving  complexity  of 
the  operating  environment  with  a  continuous  increase  of  the  number  of  actors,  the  force 
development  decisions  are  inherently  complex.  Military  organizations  are  simultaneously  facing  a 
high  rate  of  technological  evolution,  a  less  polarized  world  with  many  relevant  state  and  non-state 
actors,  and  an  increasingly  complex  theatre  of  operations.  These  factors  combined  with  the 
increasing  complexity  of  the  organization  itself  complicate  the  identifications  of  issues  and 
therefore  the  identification  of  the  best  direction  to  orient  force  development. 

Results:  The  current  report  reviews  “organization  development”  best  practices.  The  term 
“organization  development”  has  been  used  in  scientific  literature  to  cover  all  activities  performed 
in  support  of  evolving  or  modifying  an  organization.  This  literature  review  identified  three 
primary  organization  interfaces  that  require  close  scrutiny  for  an  effective  organization 
development:  the  organization-environment  interface;  the  group-to-group  interface,  and  the 
individual-organization  interface.  The  identified  best  practices  are  then  used  as  a  basis  to  review 
and  identify  limitations  of  the  current  force  development  activities  performed  within  the  Chief  of 
Force  Development  (CFD)  organization.  It  is  shown  that  the  individual-organization  interface  is 
largely  overlooked  by  the  current  approach  and  would  need  a  closer  integration  with  other  force 
development  activities.  This  aspect  of  organization  development  is  used  to  recommend  specific 
human  views  within  the  Department  of  National  Defence  Architecture  Framework.  The  report 
also  includes  specific  recommendations  with  regards  to  improving  the  Concept  Development  and 
Experimentation  (CD&E)  contributions  to  force  development.  CD&E  plays  an  essential  role  in 
ensuring  the  right  orientation  of  the  force  development  activity  and  in  reducing  the  risk  associated 
to  the  development  of  new  capabilities. 

Significance:  The  best  practices  described  in  the  current  report  provide  indication  on  ways  to 
improve  the  current  force  development  approach  within  the  Canadian  Forces.  In  particular,  the 
report  recommends  a  closer  integration  of  lessons  learned  and  the  inclusion  of  the  individual- 
organization  interface  within  force  development  activities.  The  sections  focusing  on  CD&E 
indicate  how  to  best  use  these  instruments  to  effectively  support  force  development  activities.  It  is 
hoped  that  the  arguments  provided  can  help  better  realign  some  of  these  activities. 

Future  plans:  A  more  detailed  Concept  Development  document  is  being  written  at  the  Canadian 
Forces  Warfare  Centre.  This  detailed  document  will  complement  the  current  report  as  well  as  the 
GUIDEx  which  provides  details  on  defence  experimentation. 
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Joint  Concept  Development  and  Experimentation:  A  Force 
Development  Perspective 

Dave  Allen;  DRDC  CORA  TM  2012-036;  R  &  D  pour  la  defense  Canada  -  CARO; 
fevrier  2012. 

Contexte:  Les  activites  de  developpement  des  forces  jouent  un  role  important  lorsqu’il  s’agit  de 
faire  evoluer  les  Forces  canadiennes  de  sorte  qu’elles  s’adaptent  aux  exigences  des  missions 
futures.  L’environnement  operational  devenant  de  plus  en  plus  complexe  en  raison  de 
1’ augmentation  constante  du  nombre  d’acteurs,  il  en  de  meme  pour  les  decisions  relatives  au 
developpement  des  forces.  Les  organisations  militaires  sont  confrontees  simultanement  a  une 
rapide  evolution  technologique,  a  un  monde  moins  polarise  comptant  de  nombreux  acteurs 
etatiques  et  non  etatiques  pertinents  et  un  theatre  d’ operations  de  plus  en  plus  complexe.  Ces 
facteurs  combines  a  la  plus  grande  complexity  de  1’ organisation  en  tant  que  telle  rendent  plus 
difficile  le  recensement  des  enjeux  et,  par  le  fait  meme,  la  determination  de  la  meilleure 
orientation  a  donner  au  developpement  des  forces. 

Resultats:  Le  rapport  actuel  porte  sur  les  meilleures  pratiques  en  matiere  de  «  developpement 
organisationnel  ».  Le  terme  «  developpement  organisationnel  »  est  utilise  dans  les  textes 
scientifiques  pour  designer  toutes  les  activites  menees  a  l’appui  de  revolution  ou  de  la 
modification  d’une  organisation.  Dans  le  cadre  de  cet  examen  de  la  documentation,  on  a  releve 
trois  principales  interfaces  organisationnelles  qu’il  faut  etudier  de  plus  pres  pour  un 
developpement  efficace  :  1’ interface  entre  F  organisation  et  le  milieu;  1’  interface  entre  deux 
groupes  et  F interface  entre  l’individu  et  F organisation.  On  se  fonde  ensuite  sur  les  meilleures 
pratiques  recensees  pour  examiner  et  reconnaitre  les  limites  des  activites  actuelles  de 
developpement  des  forces  menees  au  sein  de  1’ organisation  du  Chef  -  Developpement  des  forces 
(CDF).  II  est  demontre  que  l’approche  actuelle  ne  tient  pas  beaucoup  compte  de  l’interface  entre 
l’individu  et  F organisation  et  que  cette  demiere  devrait  etre  davantage  integree  aux  autres 
activites  de  developpement  des  forces.  On  part  de  cet  aspect  du  developpement  des  forces  pour 
recommander  Fapport  de  certains  points  de  vue  humains  dans  le  Cadre  d’ architecture  du 
ministere  de  la  Defense  nationale.  Le  rapport  comprend  aussi  certaines  recommandations 
concemant  F  amelioration  des  contributions  de  F  elaboration  et  experimentation  des  concepts 
(EEC)  au  developpement  des  forces.  L’EEC  est  essentielle  pour  bien  orienter  Factivite  de 
developpement  des  forces  et  pour  reduire  les  risques  associes  au  developpement  de  nouvelles 
capacites. 

Importance:  Les  meilleures  pratiques  decrites  dans  le  present  rapport  presentent  des  fa9ons 
d’ameliorer  Fapproche  adoptee  actuellement  dans  les  Forces  canadiennes  en  matiere  de 
developpement  des  forces.  En  particulier,  le  rapport  recommande  une  meilleure  integration  des 
le9ons  retenues  et  F  inclusion  de  F  interface  entre  Findividu  et  F  organisation  dans  les  activites  de 
developpement  des  forces.  Les  sections  portant  sur  l’EEC  demontrent  la  meilleure  fa9on  d’utiliser 
ces  instruments  pour  appuyer  efficacement  les  activites  de  developpement  des  forces.  II  est  a 
esperer  que  les  arguments  presentes  contribueront  a  mieux  harmoniser  certaines  de  ces  activites. 
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Recherches  futures:  Un  document  plus  detaille  sur  le  developpement  des  concepts  est  en  voie 
d’elaboration  au  Centre  de  guerre  des  Forces  canadiennes.  Le  document  en  question  completera 
le  rapport  actuel  ainsi  que  le  GUIDEx  qui  foumit  des  precisions  sur  le  processus 
d’ experimentation  dans  le  domaine  de  la  defense. 
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1  Introduction 


1.1  Background 

Contemporary  organizations  are  facing  an  environment  evolving  at  an  unprecedented  pace.  New 
technologies  are  emerging  continuously  and  an  increasing  number  of  actors  are  capable  of  having 
political  and  economic  ramifications  at  the  international  level.  In  response  to  these  changes, 
organizations  must  continuously  evolve  and  adjust.  In  fact,  empirical  data  indicate  that  even  in  a 
relatively  stable  environment  a  certain  level  of  organization  evolution  tends  to  increase  the 
organization  stability  [1].  All  these  factors  clearly  highlight  the  importance  of  organization 
development. 

From  a  military  perspective,  organizations  are  simultaneously  facing  a  high  rate  of  technological 
evolution,  a  less  polarized  world  with  many  relevant  state  and  non-state  actors,  and  an  increasing 
number  of  armed  and  unarmed  (aid  and  development)  organizations  operating  within  their  Area 
of  Operations.  In  the  past,  military  force  development  was  largely  performed  in  response  to  the 
evolution  of  the  main  opponent  capability.  However,  current  military  development  must  consider 
a  much  larger  variety  of  possible  opponents  and  keep  track  of  a  large  number  of  new 
technologies,  continuously  evolving.  Within  this  context,  concept  development  and 
experimentation  are  important  activities  to  ensure  appropriate  guidance  to  military  force 
development  and  to  reduce  the  time  required  for  the  acquisition  of  new  capabilities. 

1 .2  Aim  of  the  Report 

The  aim  of  this  report  is  to  review  organization  development  best  practices  to  establish  references 
against  which  the  current  joint  force  development  can  be  assessed.  These  best  practices  consider 
both  the  aspects  and  the  steps  that  should  be  part  of  the  organization  development  activities. 
Three  aspects  appear  particularly  important  to  organization  development  activities:  the 
organization-environment  interface,  the  group-to-group  interface,  and  the  individual-organization 
interface.  In  addition,  four  steps  are  essential  to  any  organization  development  procedures: 
identifying  existing  or  upcoming  issues,  identifying  directions  in  which  to  develop  the 
organization,  implementing  the  required  modifications,  and  assessing  the  impact  of  the 
implemented  changes.  The  three  aspects  and  four  steps  are  used  to  establish  a  framework  on 
which  to  compare  the  current  joint  force  development  process.  It  is  found  that  the  current  force 
development  approach  should  strengthen  its  current  focus  on  the  individual-organization  interface 
and  should  also  better  integrate  joint  lessons  learned.  The  identified  aspects  of  organization 
development  are  also  used  to  recommend  specific  human  views  for  inclusion  within  the 
Department  of  National  Defence  Architecture  Framework  (DNDAF). 

The  second  half  of  the  report  provides  additional  recommendations  with  regards  to  the  Concept 
Development  and  Experimentation  (CD&E)  activities.  These  activities  play  particular  importance 
in  ensuring  the  adequacy  of  proposed  force  development  solutions  and  reducing  the  risk 
associated  with  the  implementation  of  these  solutions.  Considering  the  three  aspects  and  four 
steps,  it  is  indicated  how  the  CD&E  activities  can  be  beneficial  to  force  development. 
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This  report  will  provide  useful  information  and  guidelines  to  the  broader  military  community 
recently  involved  in  force  development  and/or  CD&E  activities  (Directorate  of  Military 
Capability  Management,  the  Cyber  Task  Force,  as  well  as  the  Joint  Information  and  Intelligence 
Fusion  Capability  (JIIFC)  project,  the  Collaborative  Operational  Planning  System  (COPS) 
project,  the  Chief  of  Defence  Intelligence  (CDI),  the  Strategic  Joint  Staff  (SJS),  1st  Canadian 
Division,  the  Chief  of  Programs  (CProg),  the  Canadian  Expeditionary  Forces  Command 
(CEFCOM),  Canada  Command  (CanadaCOM),  and  the  Canadian  Operational  Support  Command 
(CANOSCOM)). 

1.3  Outline 

The  report  is  divided  as  follows: 

•  Section  2  describes  primary  organization  development  models  used  in  management 
science.  This  first  section  provides  a  broader  context  on  force  development  and  its 
purposes.  The  described  model  is  used  to  build  the  framework  that  is  used  in  the 
following  section  to  review  the  current  force  development  methodology  used  within  the 
Canadian  Forces. 

•  Section  3  discusses  the  current  force  development  process  as  performed  by  the  Chief  of 
Force  Development  (CFD)  organization.  This  section  focuses  on  the  central  item  of  the 
force  development  approach  which  is  based  on  a  Capability-Based  Planning  (CBP) 
methodology.  This  approach  is  compared  with  the  scientific  model  of  organization 
development  introduced  in  section  2.  This  comparison  is  of  particular  interest  within  the 
context  of  the  on-going  activities  to  develop  a  new  version  of  the  Strategic  Capability 
Roadmap  (SCR).  This  section  highlights  the  limitations  of  the  current  force  development 
activities  and  provides  recommendations  to  more  closely  align  these  activities  to  existing 
best  practices. 

•  Section  4  focuses  the  discussion  on  concept  development  activities.  The  specific 
contributions  of  concept  development  are  discussed  and  recommendations  are  made  to 
ensure  the  effectiveness  of  these  contributions  considering  the  various  aspects  that  force 
development  should  include. 

•  Section  5  is  similar  to  section  4  with  the  exception  that  the  focus  is  now  on 
experimentation  activities  and  its  contributions  to  force  development.  This  section  is 
largely  based  on  The  Technical  Cooperation  Program  (TTCP)  Guide  for  Understanding 
and  Implementing  Defense  Experimentation  (GUIDEx).  [27] 

•  Section  6  concludes  and  provides  the  reasoning  for  a  more  integrated  approach  between 
the  various  force  development  activities. 
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2  Organization  Development  Sciences 


2.1  A  System-of-Systems  Approach 

Within  the  context  of  management  science,  organization  development  is  defined  as  the  set  of 
activities  performed  for  finding  and  implementing  ways  to  change  the  organization  from  its 
current  state  to  a  better-developed  state.  This  entails  four  types  of  activities1:  1)  identifying 
existing  or  upcoming  (i.e.,  future)  issues;  2)  identifying  directions  in  which  to  develop  the 
organization;  3)  implementing  the  required  modifications;  and,  4)  assessing  the  impact  of  the 
performed  changes.  This  seemingly  simple  sequence  of  tasks  is  rendered  complex  by  the  plethora 
of  subjective  views  that  impact  each  task.  In  particular,  one  should  note  that  the  identification  of 
issues  is  matter  of  perception.  If  one  subjectively  perceives  an  activity  or  its  outputs  more 
difficult  to  complete  than  expected,  then  he  will  associate  specific  issues  to  this  activity. 
Therefore,  issues  are  subjective  and  relative  to  expectations. 

Force  development  (FD)  is  becoming  more  complicated  due  to  the  increasing  complexity  of 
organizations.  In  particular,  the  organizational  boundaries  are  becoming  blurry.  Some  firms,  such 
as  Walmart,  are  becoming  more  interconnected  with  their  merchandise  providers  and  in  some 
cases  sharing  databases  and  infrastructure.  Also,  various  organizations  have  external  contractors 
closely  embedded  within  their  own  units  making  the  distinction  between  internal  and  external 
organization  aspects  less  obvious.  All  these  factors  increase  the  organization’s  complexity  and 
complicate  the  identification  of  the  root  cause  for  identified  issues  and  the  prediction  of  the 
impact  of  specific  changes. 

Organization  development  was  initially  done  primarily  by  behavioural  scientists  and  focused  on 
improving  staff  training.  However,  researchers  have  highlighted  the  need  for  a  broader  approach 
to  organization  development  (see  references  [2]  and  [3]).  In  particular,  Lawrence  and  Lorsch  have 
shown  the  importance  of  having  a  systemic  approach  to  organization  development.  They 
suggested  looking  at  organizations  from  three  different  perspectives  (see  Figure  1  where  the 
organization  is  depicted  by  the  large  black  box  within  a  conceptual  space): 

1.  The  group-to-group  interface  (shown  in  green  in  Figure  1); 

2.  The  individual-organization  interface  (shown  in  red  in  Figure  1  and  indicating  the  boundary 
between  the  organization  and  the  various  individuals  contributing  to  it);  and, 

3.  The  environment-organization  interface  (shown  in  blue  in  Figure  1  and  indicating  the 
boundary  between  the  organization  and  the  external  environment  -  the  term  “environment”  is 
used  to  refer  to  all  entities  external  to  the  organization  and  the  individuals  contributing  to  it). 

These  three  perspectives  reflect  crucial  and  essential  elements  for  organization  development:  the 
necessity  for  each  unit  to  have  characteristics  consistent  with  its  task  and  for  unity  of  effort;  the 
necessity  for  satisfactory  inducements  to  ensure  individuals  agree  and  are  motivated  to 
contributing  to  the  organization;  and,  the  necessity  of  developing  an  effective  relationship  and 
transaction  with  the  environment. 


1  The  four  identified  steps  are  similar  to  the  four  steps  of  the  OODA  loop:  Observe,  Orient,  Decide,  Act. 
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Note  that  within  the  Lawrence-Lorsch  model,  individuals  are  external  to  the  organization;  all 
individuals  exist  outside  the  organization  highlighting  the  fact  that  any  long  term  organization 
planning  needs  to  consider  the  transient  nature  of  the  individuals  and  the  associated  need  for 
recruitment  and  transfer  of  knowledge.  Only  the  individuals’  contributions  to  the  organization  are 
integral  to  it. 
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Figure  1:  Lawrence  and  Lor sch  organization  model 


2.2  Environment-Organization  Interface 

The  Lawrence-Lorsch  model  provides  a  large  number  of  factors  that  can  be  considered  and 
modified  in  response  to  an  organization  development  need.  At  the  environment-organization 
interface,  staff  must  secure  and  process  deals  as  well  as  negotiate  the  terms  of  exchange  of 
tangible  goods  and  less  tangible  services  of  many  kinds.  However,  this  interaction  with  the 
environment  might  vary  between  organizational  units.  Some  units,  such  as  an  R&D  unit,  might 
have  to  deal  with  a  very  uncertain  environment  with  emerging  new  technology  that  can  disrupt 
the  organization’s  ability  to  perform  within  its  environment.  Lawrence  and  Lorsch  recommend 
the  consideration  of  four  measurable  features  to  allow  each  unit  to  adequately  adjust  to  its 
environment: 

1.  The  degree  of  reliance  on  formalized  rules  and  formal  communication  channel  within  the 
unit; 

2.  The  time  horizon  of  managers  and  professionals  in  the  groups; 

3.  Their  orientation  toward  goals,  either  diffuse  or  concentrated;  and 
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4.  Their  interpersonal  style,  either  relationship  or  task  oriented. 

If  the  environment  is  relatively  stable,  the  necessary  information  can  be  handled  through  the 
traditional  superior- subordinate  channels,  which  may  be  few  and  constricted  but  are  less  subject 
to  error  and  relatively  inexpensive.  Fairly  short  time  horizons  are  adequate  to  take  account  of  the 
reactions  of  the  environment  to  the  firm’s  action.  The  interface  can  be  designed  on  a 
straightforward  task-oriented  approach. 

On  the  other  hand,  to  deal  with  an  uncertain  and  rapidly  changing  sector  of  the  environment, 
more  points  of  contact  and  a  flatter  organization  must  be  employed.  Formal  rules  cannot  be 
formulated  that  will  be  suitable  for  any  appreciable  time  period.  More  of  an  all-to-all 
communication  pattern  is  indicated,  which  can  keep  environmental  clues  moving  throughout  the 
unit  for  interpretation  at  all  points  instead  of  just  through  superior-subordinate  channels.  A  longer 
time  orientation  is  usually  needed.  The  growth  of  this  more  complex  and  sophisticated 
communication  network  is  fostered  by  an  interpersonal  style  that  emphasizes  building  strong 
relationships  rather  than  just  accomplishing  the  task,  per  se. 

2.3  Individual-Organization  Interface 

The  crucial  question  at  the  individual-organization  interface  is  how  individual  contributors  can  be 
induced  to  perform  their  defined  activities.  How  can  we  motivate  individuals  to  make  the 
contributions  to  the  organization  purpose  that  is  required  of  them?  How  can  the  organization 
channel  and  control  the  behaviour  of  individual  contributors  in  the  desired  direction?  These 
questions  require  a  look  at  what  motivates  individuals.  Schein  identified  three  important  motives 

[4]2: 

•  Need  for  achievement:  Need  for  competitive  success  measured  against  a  personal  standard  of 
excellence. 

•  Need  for  affiliation:  Need  for  warm,  friendly,  compassionate  relationship  with  others. 

•  Need  for  power:  Need  to  control  or  influence  own  activities  and  possibly  those  performed  by 
others. 

Schein  grouped  these  motives  under  a  global  need  for  problem-solving  (or  as  termed  by  White,  a 
need  for  a  sense  of  competence  or  mastery  [6]).  As  the  individual  system  strives  to  solve 
problems,  certain  behaviours  turn  out  to  be  consistently  rewarding;  that  is,  they  provide  solutions 
to  the  problems  the  individual  faces.  Over  time,  as  some  behavioural  patterns  (patterns  of  success, 
control,  friendly  relationship,  etc.)  are  consistently  rewarding,  the  individual  learns  to  rely  on 
them. 

The  problems  and  challenges  perceived  by  an  individual  in  the  organizational  setting  is  shaped  by 
the  expectations  of  others,  by  the  nature  of  the  task  which  the  individual  is  required  to  perform 


2  More  recently,  Daniel  H.  Pink  came  to  a  similar  conclusion  with  regards  to  the  key  motives  impacting  an 
employee’s  contributions  [5].  He  used  the  term  autonomy,  mastery  and  purpose  to  summarize  the  motives. 
Respectively,  according  to  Pink,  these  three  terms  expressed  the  employee’s  need  to  choose  what  and  how 
tasks  are  completed,  the  need  to  become  adept  at  an  activity,  and  the  desire  to  improve  the  world.  These 
three  motives  have  a  close  similarity  to  Schein’ s  needs  for  power,  achievement,  and  affiliation. 
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and  by  the  formal  organization  variables,  such  as  supervisory  style,  rewards  and  punishments, 
rules,  control  procedures,  etc.,  with  which  he  is  confronted.  The  individual  system,  others’ 
expectations,  the  task  and  formal  organization  variables  all  help  shape  the  individual’s  view  of 
what  is  expected  of  him  by  the  organization. 

This  conceptual  scheme  highlights  two  opportunities  for  developing  the  individual-organization 
interface: 

•  Developing  the  individual  systems  to  make  them  more  consistent  with  the  rewards  available 
from  task  and  organizational  factors. 

•  Alter  the  task  or  organizational  variables  so  that  they  will  afford  a  higher  expectation  of  need 
satisfaction. 

Such  an  approach  to  developing  the  individual-organization  interface  means  placing  a  heavy 
emphasis  on  diagnosis.  We  need  to  identify  in  what  activity  groups  problems  exist.  What  are  the 
attributes  of  these  jobs  and  what  behaviour  leads  to  effective  performance?  What  individual 
system  characteristics  are  most  likely  to  motivate  individuals  to  perform  this  sort  of  task 
effectively?  Are  the  expectations  of  others,  the  organizational  relationships,  rewards,  and  controls 
likely  to  motivate  individuals  to  perform  this  sort  of  task  effectively?  Are  the  expectations  of 
others,  the  organization  relationships,  rewards,  and  controls  likely  to  work  toward  increasing  the 
expectation  of  need  attainment,  or  will  they  operate  in  the  opposite  direction? 

2.4  Group-to-Group  Interface 

To  manage  a  large  complex  organization,  it  is  common  to  divide  it  into  units  that  can  function 
nearly  independently  of  each  other.  This  division  reflects  the  common  view  that  the  grouping  of 
specialized  units  will  outperform  a  team  of  generalists  with  regards  to  the  quality  of  the  task 
output  and  of  their  productivity. 

Three  aspects  need  to  be  considered  when  considering  the  division  of  the  organization  into 
modular  units: 

•  The  resources  (human,  authorities,  responsibilities,  etc.)  assigned  to  each  units; 

•  The  level  of  differentiation  of  the  organization  into  units; 

•  The  level  of  integration  between  units  and  the  communication  and  technical  tools  supporting 
this  integration. 

The  level  of  differentiation  within  an  organization  depends  upon  what  internal  characteristics 
each  group  must  develop  to  carry  out  planned  transactions  with  its  assigned  part  of  the 
environment.  More  specifically,  it  depends  primarily  upon  the  extent  to  which  the  certainty  of 
information  within  the  various  parts  of  the  environment  is  similar  or  different.  The  differentiation 
of  the  organization  into  units  ensures  a  certain  modularity  reinforcing  the  organization  robustness. 

The  objective  of  organization  development  efforts  at  the  group-to-group  interface  is  to  achieve 
collaboration  or  integration  between  the  groups  of  specialized  contributors  so  that  they  can  make 
a  coordinated  effort  toward  total  organization  goals,  while  still  working  effectively  at  managing 
the  transactions  with  their  particular  segment  of  the  environment. 
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Integration  between  groups  deals  with  two  aspects:  which  units  are  required  to  work  together  and 
how  tight  the  requirement  is  for  interdependence  among  them.  When  units  are  highly 
differentiated,  it  is  more  difficult  to  achieve  integration  among  them  than  when  the  individuals  in 
the  units  have  similar  ways  of  thinking  and  behaving.  As  a  result,  when  groups  in  an  organization 
need  to  be  highly  differentiated,  but  also  require  tight  integration,  it  is  necessary  to  develop  more 
complicated  integrating  mechanisms.  Organization  integration  research  provides  a  list  of  factors 
that  impact  on  the  level  of  integration.  Barki  and  Pinsonneault  have  recently  proposed  an 
Organization  Integration  model  [7]  which  is  characterized  based  on  the  number  of  firms  involved 
(internal:  within  a  single  organization;  external:  involving  at  least  two  firms)  and  on  the  type  of 
integration:  procedural  (integration  across  procedures)  or  functional  (integration  by  providing 
additional  functionalities).  The  Barki  and  Pinsonneault  model  also  integrates  the  type  of 
interdependency  as  introduced  by  Thomson  [8].  Three  types  of  interdependencies  were 
introduced  by  Thomson:  Pooled,  Sequential  or  Reciprocal.  Pooled  interdependency  is  the 
simplest  and  corresponds  to  the  functional  integration  where  the  various  integrated  organizations 
share  common  pooled  resources.  Sequential  interdependency  implies  that  the  output  of  one 
organization  is  used  as  input  by  the  other.  The  reciprocal  interdependency  is  the  most  complex 
and  implies  back  and  forth  procedural  interaction  between  the  organizations.  This  model  indicates 
that  an  organizational  division  to  reduce  the  need  for  procedural  integration  as  well  as  reciprocal 
integration  is  ideal.  The  model  also  provides  a  list  of  mechanisms  that  can  be  used  to  achieve  the 
required  integration: 

•  Mutual  team  adjustment; 

•  Direct  supervision; 

•  Standardization  of  output; 

•  Standardization  of  work; 

•  Standardization  of  skills  and  knowledge; 

•  Standardization  of  norms;  and, 

•  Pre-planning. 

Mutual  team  adjustment  and  direct  supervision  will  require  communication  systems  if  the  various 
units  are  not  co-located.  It  also  requires  trust  and  understanding  between  groups  and  the 
confrontation  of  conflict.  Too  often  specific  conflict-management  variables  are  not  considered, 
such  as  those  that  are  contingent  on  environmental  demands;  on  the  required  state  of 
differentiation;  and  also  on  the  design  of  appropriate  structural  devices  for  achieving  integration. 

The  Barki-Pinsonneault  model  is  not  limited  to  a  single  organization,  but  could  also  be  considered 
for  assessing  the  level  of  integration  across  different  organizations  (e.g.,  the  model  was  recently 
used  for  assessing  the  level  of  organization  integration  for  a  multi-national  operation  where 
military  units  from  different  countries  share  a  common  airspace  [9]).  For  example,  within  a 
coalition  setting  or  a  Whole-of-Govemment  approach,  the  applications  of  specific  mechanisms, 
such  as  direct  supervision,  are  limited  due  to  the  limited  accountability  enforced  between  the 
various  organizations.  Constraints  are  only  self-imposed  by  agreeing  to  specific  treaties.  These 
limitations  increase  the  reliance  on  the  other  mechanisms  to  ensure  an  adequate  integration  of  all 
organizations  involved  within  the  coalition. 
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2.5  Summary  of  Organization  Development 


The  various  elements  of  the  Lawrence-Lorsch  organization  development  model  are  summarized 
in  Figure  2. 

Not  all  factors  identified  by  the  Lawrence-Lorsch  model  can  be  as  easily  modified  to  perform 
organization  development.  Some  factors  require  a  more  fundamental  behaviour  change,  which  are 
less  easily  implemented.  Table  1  summarizes  the  list  of  possible  targets  for  changes  and  display 
their  associated  behaviour  change.  This  table  shows  that  a  modification  of  the  tools  and  of  the 
interaction  patterns  (tactics,  techniques  and  procedures)  are  the  simplest  to  implement  and  should 
be  sought  prior  to  more  disruptive  approaches.  Note  however  that  the  change  methods  do  not 
necessarily  impact  a  single  change  target.  For  example,  an  intensive  educational  program  would 
in  the  long  term  impact  an  individual’s  basic  motives.  The  association  made  within  Table  1 
should  be  considered  as  the  primary  target  associated  with  the  change  methods. 


Table  1:  Hierarchy  of  organization  development  changes 


Change  Target 

Change  Method 

Modest 

Behaviour 

Change 

V 

Fundamental 

Behaviour 

Change 

Simplifying  human  resource 
needs 

Technical  support  to  human  contributions; 
automation  of  routine  and  redundant 
tasks 

Different  interaction  patterns 

New  coordination  methods,  budgets, 
schedules,  official  channels  of 
communication;  database  integration 

Different  role  expectations 

Intensive  educational  programs;  new 
divisions  of  labour  and  authority  structure 

Different  orientations  and 
values 

New  reward  systems;  different  leadership 
styles 

Different  basic  motives 
(achievement,  power, 
affiliation) 

New  selection  criteria;  replacement  or 
incumbents;  or  major  strategy  change 

For  complex  changes  impacting  several  change  targets,  a  system-engineering  approach  (see  [10]) 
such  as  the  V-model  is  recommended  (see  Figure  3).  Note  that  the  stakeholder  requirements, 
which  are  obtained  by  focusing  solely  on  the  problem-space  corresponds  to  the  first  step  of 
identifying  issues.  These  stakeholder  requirements  might  relate  to  any  of  the  three  identified 
interfaces.  Each  interface  relates  to  different  subsystems  within  an  organization.  For  example,  the 
group-to-group  interface  is  associated  with  the  integrating  and  communication  subsystem  while 
the  individual-organization  interface  refers  to  the  reward,  supervisory,  control  procedure 
subsystems.  The  principal  benefit  of  the  Lawrence-Lorsch  model  is  to  provide  a  framework 
according  to  which  an  organization  is  subdivided  into  three  complementary  interfaces  that  can  be 
used  as  the  basis  on  which  organization  development  is  done.  The  organization  development  best 
practices  highlight  the  importance  of  the  first  two  steps  which  consist  of  identifying  existing  or 
upcoming  issues  and  identifying  directions  in  which  to  develop  the  organization.  These  two  steps 
are  essential  for  the  establishment  of  the  stakeholder  requirements  on  which  the  whole 
development  activity  depends.  The  importance  of  these  steps  is  further  supported  by  investigating 
the  reasons  for  failure  of  organization  development  activities. 
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Figure  2:  Overview  of  Lawrence-Lorsch  organization  development  model 
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Figure  3:  V- Mo  del  for  systems  engineering. 
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Table  2  provides  the  results  of  a  survey  performed  by  the  Standish  Group  in  1995  and  1996  to 
identify  the  different  reasons  for  project  failure  (see  [10]  for  a  description  of  the  survey  results). 
Although  this  survey  considered  general  projects,  it  is  certainly  applicable  to  organization 
development  projects.  The  survey  was  distributed  to  a  large  number  of  staff  members  within 
several  private  firms.  The  staff  members  were  requested  to  identify  reasons  behind  the  failure  of 
projects  in  which  they  were  involved.  The  various  identified  reasons  were  then  clustered  in 
different  categories,  for  which  the  most  frequent  ones  appeared  in  the  left  column  of  Table  2.  This 
survey  indicates  that  the  most  common  reason  for  project  failure  is  the  incomplete  set  of 
requirements.  This  result  highlights  the  importance  of  having  a  model  such  as  the  Lawrence- 
Lorsch  model  to  ensure  a  systemic  consideration  of  all  aspects  implied  by  the  organization 
development  project.  The  danger  of  “lack  of  user  involvement”  also  shows  the  importance  of  staff 
member  involvement  in  ensuring  the  right  direction  for  organization  development. 


Table  2:  Reasons  for  project  failure 


Reasons 

Percentage 

Incomplete  requirements 

13.1% 

Lack  of  user  involvement 

12.4% 

Lack  of  resources 

10.6% 

Unrealistic  expectations 

9.9% 

Lack  of  executive  support 

9.3% 

Changing  requirements/specification 

8.7% 

Lack  of  planning 

8.1% 

Did  not  need  it  any  longer 

7.5% 
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3  Force  Development 


3.1  Role  of  the  Chief  of  Force  Development 

Force  development  is  the  procedure  instituted  within  the  Canadian  Forces  to  ensure  that  the 
Canadian  military  will  be  successful  in  meeting  the  missions  of  tomorrow.  At  the  joint  level,  this 
activity  falls  under  the  purview  of  the  CFD,  whose  mandate  is  defined  as  follows: 

•  Harmonize,  synchronize  and  integrate  the  force  development  activities  of  the  Environment 
Chief  of  Staff  and  functional  Lis; 

•  Focus  on  development  of  integrated  capabilities  by  formalizing  CBP,  Capability  Management 
(CM)  and  Capability  Production  (CP)  in  a  coherent  end-to-end  Force  Development  process; 
and, 

•  Provide  a  unified  Force  Development  “drumbeat”  or  “battle  rhythm”. 

Within  this  role,  the  CFD  has  established  the  force  development  process  as  displayed  in  Figure  4. 
The  force  development  process  is  displayed  in  the  middle  part  of  the  figure.  Foreign  and  Defence 
policies  are  used  to  develop  a  picture  of  the  future  security  environment.  This  initial  work  leads  to 
the  development  of  force  development  scenarios  describing  the  likely  missions  involving  the 
future  CF  and  describing  the  context  in  which  these  missions  will  be  performed. 


_ ,  I”  Force  I  ^  Force 

^  Integrated  Force  Development  I  Generators  I  ^Employment  J 


Figure  4:  Overview  of  the  CF  force  development  process 
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3.2  Capability-Based  Planning 


Once  a  picture  of  the  future  security  environment  has  been  established,  a  CBP  process  is  initiated. 

This  process  includes  three  subsequent  phases  leading  to  the  development  of  a  SCR,  which  feeds 

into  the  Investment  Plan  (IP).  The  three  phases  consist  of  (see  Figure  5): 

•  Capability  analysis:  Investigate  and  prioritize  the  list  of  capabilities  required  to  meet  the 
missions  within  the  future  security  environment. 

•  Capability  assessment:  Compare  the  forecasted  CF  capability  with  the  required  set  of 
capabilities  based  on  the  capability  analysis  for  identifying  investment,  divestment,  and 
sustainment  (IDS)  priorities. 

•  Capability  Integration:  Develop  courses  of  action  for  meeting  the  IDS  established  priorities. 

CBP  -  Phased  Approach 


Phase  1 

Phase  2 

Phase  3 

Capability  Analysis 

Capability  Assessment 

Capability  Integration 

Mission  Analysis 

Mi-swor  Effects  Scoring 

capability  selects 

Assess  FGs  to  Capabilities 

Produce  Capability  Oufloak 
t  with  IDS  Options 

Develop  Capability 
Initiatives  based  on  IDS 
Direction 

Capability  Sc  ortig 

Identifying  MoC  Limits 

■  Analyse  IDS  Option  to 
include  (PEIR) 

KLE  Decision,  on 

Capability  Initiatives 

KL B  decision  on  IDS 

XI 

Prioritized  Capability 
Requirements 

JV 

SCR 

C  Prog  for 
inclusion  in 

IP 

Figure  5:  Overview  of  the  capability-based  planning  process 


Within  the  CBP  process,  a  capability  is  not  reducible  to  a  simple  set  of  equipment.  In  fact,  the 
force  development  staff  members  are  requested  to  incorporate  all  of  the  PRICIE  elements  (see 
Figure  6)  in  their  analysis  of  required  capabilities.  Capabilities  require  trained  and  ready 
personnel  supported  by  adequate  equipment,  infrastructure,  doctrine,  procedures,  and  Information 
Technology  systems.  All  these  various  aspects  are  supported  by  Research  and  Development 
activities,  which  contribute  to  the  development  of  the  required  capabilities. 
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Analysis 


I 

C 

I 

E 


R&D  &  OR 

Research 

Infrastructure 
&  Org 

Strategic  Force  Dev 

Concepts  /  Doctrine  & 
Collective  Training 

Concepts  &  Doctrine 

Information  & 

IT  Systems 

Decision  Support 

Equipment 
(Proc  &  Field) 

Procurement 

Manage  Infrastructure 


Strategic  Policy 


Knowledge  Mgmt 


Readiness 


Experimentation 


Collective  Training 


Figure  6:  Overview  of  the  PRICIE  Elements 


3.3  CBP  Versus  Organization  Development  Best  Practices 

The  PRICIE  approach  to  capability  development  covers  a  large  portion  of  the  organization 
development  components  proposed  by  Lawrence  and  Lorsch.  Table  3  shows  the  mapping 
between  the  various  PRICIE  elements  and  the  list  of  change  methods  proposed  by  Lawrence  and 
Lorsch  (see  Table  1).  The  Research,  Development  and  Operational  Research  aspect  of  PRICIE  is 
not  indicated  as  it  permeates  all  the  proposed  change  methods. 


Table  3:  Comparison  of  Lawrence-Lorsch  change  methods  with  PRICIE  elements 


Change  target 

Change  method 

PRICIE  Elements 

Simplifying  human 
required  contributions 

Technical  support  to  human  contributions; 
automation  of  routine  and  redundant  tasks 

IT  systems,  Equipment 

Different  interaction 
patterns 

New  coordination  methods,  budgets, 
schedules,  official  channels  of 
communication,  database  integration 

Doctrine,  collective 
training,  IT  systems 

Different  role 
expectations 

Intensive  educational  programs;  new 
divisions  of  labour  and  authority  structure 

PD  &  Leadership, 
Infrastructure  &  Org 

Different  orientations 
and  values 

New  reward  systems;  different  leadership 
styles 

PD  &  Leadership 

Different  basic  motives 
(achievement,  power, 
affiliation) 

New  selection  criteria;  replacement  or 
incumbents;  or  major  strategy  change 

PD  &  Leadership 
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Interesting  conclusions  can  be  deduced  from  this  mapping  of  the  PRICIE  elements:  changes 
involving  the  PD  &  Leadership  and  Infrastructure  &  Organization  elements  of  PRICIE  imply  a 
stronger  behaviour  change  (since  they  are  associated  with  factors  further  down  on  the  list  of 
Lawrence  and  Lorsch  scale  -  see  Table  1)  and  will  therefore  require  more  time  and  effort  to  lead 
to  an  effective  modification  of  the  organization. 

Notwithstanding  that  the  PRICIE  elements  cover  most  of  the  Lawrence  and  Lorsch  elements  for 
organization  development,  the  scenario-based  approach  used  within  the  CBP  methodology  to 
force  development  provides  only  a  limited  perspective  to  the  broader  organization  development 
proposed  by  Lawrence  and  Lorsch.  Essentially,  the  scenario-based  approach  is  largely  limited  to 
the  environment-organization  and  group-to-group  interfaces  perspective.  It  focuses  on  likely 
missions  that  will  be  required  of  the  Canadian  Forces  and  the  associated  required  capabilities  both 
for  dealing  with  the  environment  and  for  ensuring  the  right  level  of  integration  (this  focus  being 
mostly  under  the  Command,  Control,  Computer  and  Communication  (C4)  aspects  of  the 
missions).  Therefore,  the  CBP  approach  aims  at  answering  questions  such  as: 

•  What  equipment,  doctrine,  training,  and  human  resources  will  the  Canadian  Forces  need  to 
meet  the  future  threats  and  mission  requirements? 

The  situation  can  be  summarized  as  follows:  Although,  various  human  factors  are  considered 
within  the  capability  engineering  approach  (through  the  PRICIE  methodology),  the  step  which 
consist  of  “identifying  current  or  future  issues”  (identification  of  the  needs,  which  is  scenario 
based)  does  not  consider  the  individual-organization  interface. 

3.4  Capability-Based  Planning  Limitations 

Our  review  indicates  that  the  individual-organization  interface  is  largely  overlooked  by  the 
scenario  based  approach.  As  argued  by  Lawrence  and  Lorsch,  this  interface  requires  a  large 
diagnostic  effort  (identifying  the  current  status  of  the  organization  at  the  individual  level),  which 
is  not  the  main  focus  of  the  scenario-based  approach.  In  consequence,  the  current  CBP  approach 
will  not  provide  answers  to  questions  such  as: 

•  Is  there  a  healthy  diversity  of  views  across  the  various  CF  units  or  is  there  too  many 
conflicting  views  leading  to  a  waste  of  efforts? 

•  Are  the  employees  thinking  in  terms  of  simply  a  job  or  of  a  career? 

•  How  much  emotional  commitment  to  organization  goals  is  offered  and  expected? 

•  What  balance  is  struck  between  dependence  and  independence,  between  conformity  and 
creativity,  between  duty  and  self-expression? 

•  Is  the  organization  accumulating  a  reservoir  of  trained  human  assets  and  good  will,  or  is  it 
dissipating  human  resources  built  up  in  an  earlier  period? 

•  What  balance  of  efforts  respectively  goes  into  CBP,  CF  units’  integration  and  resource 
motivation  efforts?  Is  the  distribution  of  efforts  adequate? 

•  Is  the  ratio  between  reservist,  regular  forces  and  public  servant  adequate  considering  the 
motivation  and  expectation  of  each  group  respectively  to  the  organization  need? 
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These  remarks  highlight  the  fact  that  the  CBP  approach  cannot  be  the  sole  approach  to 
organization  development.  This  conclusion  might  not  be  surprising  as  other  units,  in  addition  to 
CFD,  participate  in  one  way  or  another  to  the  force  development  effort  (through  recruiting, 
training,  etc.).  However,  it  invites  additional  questions  such  as:  Can  the  CBP  process  be 
performed  independently  of  other  organization  development  activities?  Future  military  activities, 
equipment,  training  will  impact  the  individual-organization  interface  (impact  on  the  individual’s 
expectation,  on  rate  of  retention,  etc.)  and  therefore  a  broader  approach  to  force  development  may 
be  desirable.  This  conclusion  is  in  agreement  with  decision-making  best  practices  [12]  which 
indicate  that  the  impact  resulting  from  the  implementation  of  organization  changes  on  human 
motivation  are  crucial  and  should  be  considered  within  the  selection  of  possible  force 
development  options.  Therefore,  a  strategic  lesson  process  that  is  considering  the  organization  at 
large  including  the  level  of  satisfaction  among  the  CF  members  and  their  view  on  the  overall 
organization  should  be  established  to  gather  the  required  information  to  steer  the  force 
development  process  in  the  right  direction. 

3.5  DNDAF  Human  Views 

The  existing  Department  of  National  Defence  Architecture  Framework  (DNDAF  version  1.7) 
incorporates  various  views  that  can  be  used  to  guide  force  development  activities.  Apart  from  the 
Common  Views,  Strategic  Views,  and  Capability  Views  that  are  used  more  to  set  the  context,  the 
other  architecture  views  are  centered  around  four  items  and  their  interconnection  (see  Annex  A 
for  a  details  of  the  various  views):  the  people,  the  technical  system,  the  established  procedures, 
and  the  required  information. 

The  various  views  provide  much  of  the  information  discussed  within  the  Lawrence-Lorsch  model 
of  organizations.  For  example,  the  differentiation  of  activities  into  different  groups  and  the 
relationship  between  these  groups  are  described  by  the  Operational  View  (OV)  4  views. 
Similarly,  the  transactions  between  the  organization  and  the  environment  are  captured  within  the 
Capability  Scenario  Analysis  Matrix  (Capability  View  2)  that  describes  the  various  mission 
effects.  However,  even  though  the  skill  set  required  by  the  staff  members  and  their  expected 
activities  are  described  by  specific  operational  views  (respectively  the  OV-4b  and  OV-5a),  the 
individual-organization  interface  is  currently  not  captured  within  any  architecture  views.  To 
remedy  this  limitation,  it  is  suggested  to  add  Human  Views  to  the  list  of  architecture  views.  The 
description  of  the  individual-organization  interface  suggests  structuring  the  human  views  into 
three  categories  based  on  Schein  motives  (see  section  2.3):  human  achievement,  human 
affiliation,  and  human  power/leadership.  Diagrams  could  display  the  degree  of  fulfillment  of 
these  different  individuals’  needs  for  each  possible  trade  as  well  as  the  activities,  program  and 
policies  set  to  support  these  needs.  The  requirements  specified  within  these  diagrams  would  also 
generate  additional  process  models  to  be  included  within  the  operational  views  (OV-5b).  This 
generation  would  ensure  the  coherence  of  the  various  views. 
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4  Concept  Development 


4.1  Introduction 

The  previous  chapter  provided  a  review  of  the  joint  force  development  activities  using 
organization  development  best  practices.  This  review  was  limited  to  the  first  two  main  steps 
within  the  force  development  activities  (see  Figure  4):  the  development  of  scenario  based  on  the 
Future  Security  Analysis  and  the  Capability-Based  Planning  process.  However,  additional 
activities  are  performed  within  the  force  development  process:  in  particular,  concept  development 
and  experimentation  (CD&E).  This  section  focuses  on  the  contributions  of  concept  development 
to  the  organization  development  process  and  the  implications  of  organization  development  best 
practices  on  concept  development  activities. 

4.2  Concept  Definition 

Within  the  force  development  context,  a  concept  is  defined  as  a  notion  or  statement  of  an  idea, 
expressing  how  something  might  be  done  or  accomplished,  that  may  lead  to  an  accepted 
procedure.  More  precisely,  the  concepts  describe  the  method  (ways)  for  employing  military 
capabilities  (means)  to  accomplish  given  missions  (ends).  Military  concepts  inform  the  CBP 
process  by  providing  a  prescriptive  way  of  employing  future  capabilities  to  meet  future  missions. 

At  the  joint  level,  a  broad  number  of  concepts  have  been  defined  and  developed:  [13] 

•  The  Future  Security  Environment  (FSE):  A  strategic  level  concept  that  analyzes  international 
emerging  trends,  places  them  into  a  Canadian  context,  and  describes  the  future  type  of 
missions  that  the  CF  will  accomplish. 

•  Integrated  Capstone  Concept  (ICC):  A  basic,  strategic-level  Operating  Concept,  which 
provides  a  broad  description  of  how  future  forces  will  operate  across  the  assigned  range  of 
military  operations  in  accordance  with  government  direction.  It  turns  the  FSE  and  Scenarios 
into  an  analysis  of  the  operational  requirements  of  the  CF  and  the  effects  required  to  be 
strategically  relevant,  operationally  responsive  and  tactically  decisive  in  the  FSE. 

•  Integrating  Concepts  (IC):  Concepts  that  identify  the  effects  and  attributes  required  by  the 
future  CF  in  order  to  meet  a  specific  mission  or  condition  set.  They  describe  how  various 
broad  core  operational  activities  relate  and  how  they  can  be  integrated  into  a  cohesive 
operating  system  within  the  future  CF.  In  particular,  they  describe  how  Defence  must  be 
integrated  beyond  joint  and  combined  to  include  relationships  between  Defence  and  external 
organizations  and  the  level  of  interoperability  needed  with  those  organizations  to  successfully 
achieve  its  goals. 

•  Operating  Concepts  (OC):  Concepts  that  describe  the  “how”  within  the  constraints  and 
restraints  of  the  integrating  concept  and  scenario  generated  from  the  FSE.  While  the  IC 
identify  the  strategic  effects  and  force  attributes  for  future  defence  capabilities,  the  related 
operating  concepts  describe  how  those  capabilities  could  be  employed  within  a  particular 
mission  or  condition  set. 
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•  Enabling  Concepts  (EC):  Strategic,  operational  or  tactical  concepts  that  cover  the  entire  range 
of  condition  sets  where  technologies  (i.e.,  Artificial  Intelligence)  or  methodologies  (i.e., 
Comprehensive  Approach)  affect  the  nature  of  future  operations.  If  a  methodology  or 
technology  is  key  to  delivering  the  effects  described  by  a  particular  integrating  or  operating 
concept,  then  it  becomes  an  “enabler”. 

It  should  be  obvious  from  this  description  that  a  hierarchy  exists  among  the  various  concepts. 
Figure  7  illustrates  the  relationships  among  the  various  concepts.  The  ICC  uses  the  FSE  as  input 
to  identify  primary  requirements  for  the  future  CF  (explicitly  the  need  for  an  integrated, 
comprehensive,  adaptive  and  networked  force).  The  ICC  is  used  as  input  to  the  IC,  which 
identifies  the  force  attributes  required  and  describes  how  the  core  activities  will  be  integrated  to 
respond  to  each  mission  expected  of  the  CF.  The  OC  describes  how  each  identified  attribute  will 
operate  within  each  environment  (land,  air,  maritime,  space,  cyber,  and  cognitive3). 


1^  iw 


Nature  of 
Future... 
Conflicts 
Functions 
Environment 
Operations 


I 


New  Scenarios 


Existing  Scenarios 


S&T  /  Research  /  Analytical  Support 


CONCEIVE  FUTURE  CF  /  DND 


Experimentation  Support 


Figure  7:  Hierarchy  of  force  development  concepts 


4.3  Concept  Requirements 

The  broad  definition  of  concept  leads  to  a  plethora  of  possible  subordinate  or  supporting 
concepts.  Considering  the  list  of  possible  military  capabilities,  going  from  the  capability  to  sense 
changes  within  the  environment  to  the  capability  to  deliver  fires  at  a  distance  while  minimizing 
the  risk  of  fratricide  and  keeping  the  soldiers  well  protected,  one  ends  up  with  many  possible 
concepts  that  could  be  written  on  how  to  employ  each  of  these  capabilities.  Furthermore,  when 
coupled  with  the  consideration  of  the  potential  circumstances  under  which  each  capability  could 


3  More  recently  the  “cognitive”  domain  was  removed  as  one  of  the  CF  operating  domains  and  is  now 
considered  as  an  aspect  that  must  be  considered  throughout  all  of  the  operating  domains  (land,  air, 
maritime,  space  and  cyber). 
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be  employed  and  of  how  to  integrate  various  packages  of  capabilities  into  an  effective  set  to  meet 
specific  mission  requirements,  one  ends  up  with  an  even  much  larger  list  of  potential  concept 
papers.  The  resulting  question  is  how  to  select  the  concepts  worth  developing.  Scientific 
considerations  as  well  as  force  development  considerations  provide  answers  to  this  question. 

From  a  scientific  perspective,  concepts  are  introduced  to  explain  some  phenomena.  As  described 
by  Carnap  [15],  the  scientific  concept  development  proceeds  through  two  stages:  first,  the 
considered  phenomena  must  be  clarified  sufficiently  for  scientists  to  know  what  requires 
explanations;  second,  an  exact  concept  must  be  precisely  articulated.  Carnap  also  indicates  four 
criteria  according  to  which  concepts  are  to  be  judged: 

1.  Similarity  to  the  phenomena.  If  the  concept  does  not  incorporate  a  model  that  replicates 
adequately  the  phenomena  to  a  sufficient  degree,  it  cannot  fulfill  its  function.  A  perfect  match 
cannot,  however,  be  demanded,  as  phenomena  occur  in  a  more  complex  world  that  can  be 
elaborated  through  concepts. 

2.  Exactness.  Unless  the  concept  is  precise  it  does  not  fulfill  the  purpose  of  “explaining”  the 
phenomena. 

3.  Fruitfulness.  The  new  concept  should  enable  us  to  deduce  meaningful  conclusions  and 
important  insights.  One  of  the  main  benefits  should  be  to  deepen  our  understanding  of  the 
nature  of  sciences. 

4.  Simplicity.  The  concept  should  be  as  simple  as  requirements  1  to  3  permit.  Simplicity  often 
accompanies  systematic  power  of  concepts  and  aids  in  ease  of  application  and  avoidance  of 
errors  of  application. 

Within  the  list  of  Carnap  criteria,  criterion  4  is  subordinate  to  its  predecessors.  Thus  criteria  2  and 
3  take  precedence;  we  seek  useful  concepts  that  are  formulated  with  precision.  In  sciences,  the 
use  of  mathematical  formulations  ensures  the  later  requirement  of  precision  while  the  competition 
among  scientists  for  funding  encourages  useful  concepts. 

4.4  Concept  Development  Benefits  to  CBP 

Within  the  military,  concepts  are  ultimately  developed  to  indicate  new  ways  of  achieving  specific 
military  objectives  or  to  clarify  existing  ways  that  lack  rigor  or  precision.  Therefore,  concept 
development  requires  some  knowledge  of  current  doctrine.  The  required  knowledge  is 
particularly  important  for  the  development  of  joint  concepts.  Without  the  required  knowledge, 
ideas  such  as  “centralized  command  and  decentralized  control”  are  proposed  without  a  clear 
understanding  on  the  impact  such  ideas  might  have  on  the  operations  (see  [16]  as  a  reference  on 
similar  issues). 

From  a  CBP  perspective,  concept  development  supports  the  following  activities: 

•  Description  of  the  future  operating  environment  and  the  threat  that  the  CF  will  face; 

•  Identification  and  categorization  of  required  capabilities  to  meet  future  missions;  and, 
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•  Assessment  of  the  direction  in  which  the  required  capabilities  will  be  designed  and  employed 
(informs  the  Investment,  Divestment  and  Sustainment  priorities). 

Within  the  “Conceive-Design-Build-Manage-Employ”  phases  of  capability  development  and 
management,  concept  development  enters  primarily  in  the  “Conceive”  portion,  but  it  also  informs 
all  phases  including  the  “Employ”  phase  (e.g.,  it  informs  the  development  of  the  relevant 
doctrine). 

The  FSE  clearly  satisfies  the  first  CBP  demand.  Also,  the  FSE,  ICC,  and  functional  domain 
papers  provide  operating  context  to  support  the  identification  of  capabilities  (second  CBP 
demand).  In  particular,  the  ICC  highlights  the  need  for  an  integrated,  comprehensive,  adaptive, 
and  networked  force.  Similarly,  the  more  specific  Functional  Domain  papers  indicate  the 
importance  of  a  networked  force.  For  example,  the  Sustain  Functional  Domain  paper 
recommends  moving  from  a  cold-war  supply  chain  system  requiring  large  stockpiles  towards  an 
on-time  networked  logistic  system. 

From  a  systems  engineering  point  of  view,  the  concepts  developed  correspond  to  “Abstract 
Models”  used  to  develop  the  system  requirements  from  the  stakeholder  requirements  (see  Figure 
8  where  the  list  of  models  and  their  relation  with  requirements  development  is  shown).  This  type 
of  model  is  typically  used  to  provide  the  basis  for  establishing  a  common  understanding  of  the 
proposed  solution,  albeit  at  an  abstract  level.  It  can  be  used  to  explain  the  solution  concepts  to 
those  stakeholders  who  wish  to  be  assured  that  the  developers  are  moving  in  the  right  direction. 
Therefore,  it  ought  to  be  precise  with  regards  to  the  direction  entailed  by  the  proposed  concept. 


Figure  8:  List  of  models  used  in  systems  engineering 
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It  is  worth  noting  that  in  systems  engineering  the  abstract  model  does  not  appear  first  but  it  is 
actually  preceded  by  a  “Use  Model”,  which  focuses  on  describing  the  problem  space.  This  type  of 
model  does  not  appear  explicitly  within  the  CF  force  development  process  (Figure  4).  In  fact,  a 
recent  report  indicated  that  the  current  concept  development  work  lacks  an  epistemic  approach 
that  could  be  performed  by  leveraging  the  existing  lessons  learned  process  [14].  The  introduction 
of  models  that  capture  lessons  learned  would  fill  this  gap.  A  good  starting  point  for  developing 
these  models  should  include  a  review  of  the  simulation  models  developed  by  the  U.S.  Centre  for 
Army  Lessons  Learned. 

The  requirement  discussed  on  the  abstract  model  for  systems  engineering  shows  some  similarities 
with  the  Carnap  criteria.  Applying  these  criteria  to  concept  development  leads  to  the  following 
requirements: 

1 .  Realism.  The  concept  must  be  based  on  realistic  and  achievable  targets  considering  existing 
technology  and  available  resources  (human,  financial,  limited  time).  In  particular,  it  must  not 
require  omnipotent  and  omnipresent  soldiers  capable  of  performing  Herculean  duties. 

2.  Exactness.  The  concept  must  be  precise  to  be  conveyed  and  understood  by  the  broader  force 
development  community. 

3.  Fruitfulness.  The  new  concept  should  enable  us  to  draw  meaningful  conclusions  and  have 
important  insights  with  regards  to  force  development  objectives.  In  particular,  it  should  be 
balanced  by  considering  any  possible  trade-off:  a  specific  development  direction  necessarily 
entails  moving  away  from  other  development  directions. 

4.  Simplicity.  The  concept  should  be  as  simple  as  requirements  1  to  3  permit.  Simplicity  often 
accompanies  the  systematic  power  of  concepts  and  aids  in  their  ease  of  application  and 
avoidance  of  errors  in  their  application. 

The  implications  of  these  four  requirements  are  described  in  the  following  sub-sections.  Note  that 
the  need  for  realistic  and  achievable  targets  was  shown  to  be  one  important  reason  for  the  failure 
of  projects  (see  Table  2). 

4.4.1  From  Status  Quo  to  Blue  Sky  (Fruitful  but  Realistic) 

A  concept  is  a  statement  of  how  something  might  be  done.  Within  the  force  development  context, 
concepts  are  useful  by  proposing  new  ways  to  do  more  or  better  with  less.  However,  concepts 
must  be  realistic  and  indicate  the  way  for  feasible  improvements.  If  a  proposed  concept  is  too  far 
fetched,  it  will  be  considered  meaningless  and  the  concept  development  activities  will  suffer  in 
credibility.  At  the  other  extreme,  concepts  might  propose  new  ways  that  do  not  lead  to  any 
improvements  over  the  status  quo,  in  which  case,  it  would  lead  to  changes  for  the  sake  of 
changes.  The  benefits  expected  from  the  implementation  of  the  concept  must  be  explicit.  The 
ideal  situation  is  for  the  concept  to  propose  changes  that  are  at  least  asymptotically  reachable 
within  the  next  10  to  20  years. 

The  impact  of  a  conceptual  change  on  the  force  effectiveness  over  a  period  of  time  is  illustrated  in 
Figure  9  (the  dashed  line  represents  the  anticipated  long  term  force  effectiveness  and  the  curve 
represents  the  actual  increase  of  effectiveness  obtained  through  the  implementation  of  the 
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concept).  The  concept  development 
work  is  the  first  stage  which  leads  to 
some  recommended  changes,  which  are 
implemented  at  a  specific  time 
(“Introduction”  in  Figure  9).  Following 
some  required  education,  the  changes 
lead  to  some  improvement  to  the  force 
effectiveness.  This  improvement  will  not 
be  as  good  as  a  “Blue  Sky”  view  would 
presume,  but  should  be  significantly 
better  than  the  status  quo. 

The  requirement  for  feasible  concepts 
implies  that  one  is  aware  of  the  natural 
limitations  relevant  to  the  considered 
concepts.  On  the  physical  side,  there 
exist  various  natural  limitations  such  as 
the  impossibility  to  go  faster  than  the 
speed  of  light  (“c”),  the  indivisibility  of 

electronic  charges  below  the  electron  charge  (“ e ”)  and  the  impossibility  to  reach  into  time  and 
space  scale  below  the  Plank  scale  (“/z”).  These  are  very  fundamental  limitations.  Similarly,  on  the 
electronic  side,  there  exist  various  limitations  based  on  current  technology:  magnetic  recording  of 
density  higher  than  ~1  TB/in2;  magnetic  reading  and  writing  rate  larger  than  5  GHz;  etc.  It  is  also 
possible  to  estimate  limitations  to  Random  Access  Memory  and  channel  capacity  based  on 
current  and  foreseen  technology. 


Figure  9:  Ideal  concept  effectiveness  in  view  of 
feasibility 


However,  much  less  is  known  with  regards  to  natural  human  limitations.  In  many  ways,  some  of 
these  limitations  are  still  unknown  today  and  limitations  considered  valid  in  the  past  have  been 
proven  wrong.  A  draft  paper  is  currently  underway  to  identify  factors  impacting  on  human 
cognitive  limitations  [17]. 


4.4.2  Concept  Precision  (Degrees  of  Exactness) 

An  essential  criterion  proposed  by  Carnap  is  exactness.  This  criterion  ensures  that  the  concept  is 
defined  with  sufficient  precision  for  an  observer  to  determine  with  a  high  level  of  confidence 
whether  a  concept  is  or  is  not  implemented  when  observing  the  operators  performing  some  tasks. 
Therefore,  the  concept  description  must  be  explicit  about  what  it  entails  and  also  what  it  forbids. 

However,  it  might  be  difficult  in  specific  circumstances  to  determine  if  a  concept  is  being 
implemented  or  not.  For  this  reason,  it  is  useful  to  define  a  scale  that  measures  the  degree  of 
achievement  of  the  concept.  This  scale  can  be  continuous  or  discrete,  such  as  the  scale  that  was 
developed  for  Network-Enabled  Capability  by  the  NATO  panel  SAS-065  [18].  This  scale  should 
be  simple  enough  to  be  communicated  easily  and  should  also  pass  the  test  of  clairvoyance  [12]. 
This  test  stipulates  that  a  clairvoyant  who  could  foresee  the  consequences  of  a  specific 
implementation  of  the  concept  with  no  uncertainty  shall  be  able  to  un-ambiguously  assign  a  score 
to  the  outcome  of  the  implementation. 
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Any  developed  concept  should  be  precise  enough  to  convey  the  rationale  supporting  the  expected 
improved  effectiveness  implied  by  the  concept.  In  addition,  the  concept  description  should 
provide  the  motivation  and  all  assumptions  that  led  to  the  proposed  concept.  In  ideal 
circumstances,  the  proposed  concept  would  answer  identified  lessons  from  past  operations.  It 
could  also  be  based  on  best  practices  or  from  analogy  with  concepts  employed  by  private  firms  or 
other  organizations.  The  identification  of  the  assumptions  supporting  the  considered  concept  is 
essential  to  ensure  that  the  validating  experiments  performed  to  assess  the  increase  effectiveness 
obtained  by  implementing  the  concept  are  adequately  designed. 

4.4.3  No  Free  Lunch  (Degrees  of  Fruitfulness) 

In  the  area  of  machine  learning  (statistical  inference)  and  optimization  algorithms,  the  “No  Free 
Lunch”  theorem  has  been  developed  ([19],  [20]).  This  theorem  argues  that  there  exists  no  generic 
algorithm  that  would  be  the  best  at  solving  all  types  of  optimization  problems.  The  same  is  likely 
true  for  concepts,  which  can  be  perceived  as  an  “organizational  algorithm”  -  a  description  of  the 
method  of  how  the  organization,  viewed  as  a  system,  is  to  perform  specific  tasks4. 

Considering  the  concept  of  network-enabled  capability,  as  an  example,  the  no-free  lunch  theorem 
would  indicate  that  although  optimal  within  specific  circumstances,  possible  trade-offs  imply 
varying  benefits  under  varying  circumstances.  Limitations  and  risk  associated  with  the  network- 
enabled  concept  were  reviewed  in  [21].  In  particular,  this  reference  highlights  the  risk  that  the 
network-enabled  concept  will  lead  to  a  Force  too  dependent  on  technology  and  an  over-confident 
Commander  making  decisions  based  on  an  over-simplified  picture  of  the  battlespace. 

Applying  the  no-free  lunch  theorem  to  the  context  of  concept  development  highlights  the  need  to 
consider  and  make  explicit  all  possible  trade-offs  of  new  concepts.  Only  once  these  trade-offs  are 
explicit  can  someone  identify  possible  mitigating  factors.  If  too  many  trade-offs  are  foreseen,  this 
might  indicate  that  the  process  required  to  enable  the  implementation  of  the  concept  might  be  too 
cumbersome  or  otherwise  too  risky.  However,  a  definitive  assessment  of  concepts  is  only  possible 
through  experimentation  -  the  same  applies  to  mathematical  algorithms  where  the  no-free  lunch 
theorem  has  been  developed  (see  reference  [22]). 

Figuring  out  the  consequences  of  the  implementation  of  a  concept  and  the  associated  possible 
trade-offs  are  often  the  most  difficult  aspects  required  to  assess  a  concept  (which  might  explain 
why  the  lack  of  completeness  of  all  requirements  is  often  the  most  frequent  reason  for  project 
failure  as  indicated  in  Table  2).  Evangelos  Triantaphyllou  identified  the  following  items  as  the 
main  difficulties  encountered  when  making  strategic  decisions  [12]: 

•  Figuring  out  what  aspects  are  important  in  evaluating  the  consequences  of  a  decision; 

•  Determining  the  relative  importance  of  the  different  aspects  of  the  consequences;  and, 

•  Dealing  with  the  uncertainty  about  what  consequences  will  result  within  a  specific  context. 

When  dealing  with  new  concepts,  one  might  have  to  accept  a  large  level  of  uncertainty  with 
regards  to  their  consequences.  Past  experiments  performed  at  the  Canadian  Forces  Warfare 


4  An  algorithm  is  a  method  for  solving  a  problem,  which  is  similar  to  Schein’s  view  of  what  motivates 
individuals  to  contribute  to  an  organization  [4]. 
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Centre  have  clearly  shown  that  unexpected  and  unforeseen  results  often  arise  when  technical 
systems  are  tested  on  a  larger  scale  in  conjunction  to  existing  systems. 


4.4.4  From  Simple  to  Complex  (Degrees  of  Simplicity) 

As  suggested  by  Carnap,  a  concept  shall  be  as  simple  as  possible  while  satisfying  the 
requirements  of  exactness  and  fruitfulness.  Even  though  the  requirement  of  exactness  and 
fruitfulness  might  lead  to  an  over-complicated  concept,  it  might  still  be  possible  to  maintain  some 
form  of  simplicity  by  building  the  concepts  hierarchically  using  simpler  ones. 

Such  a  hierarchy  of  concepts  was  used  within  the  U.S.  Joint  Forces  Command  Multinational 
Experiment  4  (MNE4)  to  study  the  concept  of  knowledge  management  [23].  The  conceptual  work 
started  by  developing  a  well-grounded  definition  of  knowledge.  Using  the  concept  of  proposition 
and  interpretation  as  defined  in  predicate  calculus  [24],  knowledge  was  defined  as  the 
interpretation  of  a  proposition  to  which  is  associated  a  degree  of  belief.  Based  on  this  definition,  a 
list  of  knowledge  activities  was  defined  as  all  possible  types  of  activities  which  could  impact  on 
an  individual’s  knowledge.  Finally,  knowledge  management  was  defined  as  the  set  of  processes 
that  govern  and  facilitate  the  knowledge  activities. 

This  construct  for  defining  knowledge  management  provided  a  formal  definition  on  which 
specific  metrics  were  built  to  assess  the  knowledge  management  performed  during  the  MNE4. 
Such  precise  definition  is  required  to  ensure  an  appropriate  assessment  of  concepts  within  a 
campaign  of  experimentation.  It  also  allowed  a  systemic  assessment  of  the  proposed  concept  by 
ensuring  that  all  aspects  of  knowledge  management  were  covered  within  the  assessment. 

4.5  Concept  Categories 

The  various  concepts  should  cover  all  interfaces  identified  within  the  Lawrence-Lorsch  model  to 
ensure  that  all  organization  aspects  are  adequately  designed  and  improved.  However,  the  current 
set  of  concepts  (Integrated  Capstone  Concept,  Integrating  Concept,  Operating  Concept,  and 
Enabling  Concept)  includes  either  large  encompassing  concepts  such  as  the  ICC  or  mission 
specific  concepts.  As  with  the  Capability-Based  Planning,  the  set  of  concepts  does  not  cover  the 
individual-organization  interface.  In  fact,  there  is  currently  no  specific  set  of  concepts  to  handle 
strategic  transformation,  business  transformation,  personnel  career  management,  or  other  daily 
management  needs. 

Therefore,  organization  development  best  practices  suggest  revising  the  list  of  families  of 
concepts. 
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5  Experimentation 


5.1  Experiment  Definition 

The  term  experiment  comes  from  the  Latin  word  ‘Experiri’,  which  means  “to  try”.  This  definition 
describes  precisely  the  general  aim  of  experiments  which  consists  of  learning  through  explicit 
trials.  This  aim  is  the  principle  aspect  which  makes  experimentation  intrinsically  different  from 
exercises:  while  within  an  exercise  operators  are  trained  according  to  established  doctrine,  during 
an  experiment  new  and  unproven  methods  are  tested.  Experimentation  is  the  only  possible 
mechanism  to  firmly  answer  specific  questions  (or  hypotheses).  The  way  humans  perceive  and 
understand  their  surrounding  environment  is  limited  to  an  over  simplified  picture  and  often 
alternative  theories  can  be  proposed  to  explain  observed  phenomena.  In  many  situations, 
experiments  are  necessary  to  select  the  most  adequate  theory. 

Three  different  types  of  experiments  are  commonly  done  in  support  of  military  endeavours: 

•  Discovery  experiment:  Experiments  aiming  to  explore  the  potential  benefits  of  new 
methods  (concepts)  and  of  its  underlying  aspects  and  dependency  on  external  factors. 

•  Hypothesis  testing  experiment:  Experiments  aiming  to  advance  knowledge  by  seeking  to 
falsify  or  confirm  specific  hypotheses  as  well  as  investigating  the  limitation  of  those 
hypotheses. 

•  Demonstration  experiment:  Experiments  aiming  to  display  the  benefits  of  novel 
technologies,  organization  structure,  or  concepts  of  operations. 

Often  the  various  types  of  experiments  are  not  performed  individually,  but  within  a  larger 
campaign  of  experimentation  which  can  include  all  three  types.  As  the  campaign  of 
experimentation  evolves  the  tested  concept  is  refined  and  detailed  (see  Figure  10  from  ref.  [25]). 


Typical  Application  of  Experimentation 
in  Concept  Development 
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Figure  10:  Typical  experiment  campaign  and  its  subsequent  experiments 
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The  difference  between  hypothesis  testing  and  discovery  experiments  is  worth  clarification.  For 
all  experiments,  it  is  possible  to  write  hypotheses  or  statements  that  can  be  tested  within  the 
experiment.  However,  within  a  proper  hypothesis  testing  experiment,  the  hypotheses  correspond 
to  law-like  or  derivative  law-like  sentences  [26].  Law-like  sentences  have  four  properties5: 

1 .  They  have  universal  form; 

2.  Their  scope  is  unlimited  (they  can  obviously  be  limited  to  a  specific  subject  but  not 
contextually); 

3.  They  do  not  contain  designations  of  particular  objects  (i.e.,  all  designations  are  given  to 
generic  objects  as  opposed  to  specific  ones),  and 

4.  They  contain  only  purely  qualitative  predicates. 

A  particularity  of  law-like  sentences  is  that  they  support  counterfactual  tests  (i.e.,  what  would 
happen  if...)  and  modal  import  (i.e.,  the  law-like  sentence  delineates  what  is  necessary,  possible, 
or  impossible).  These  properties  limit  the  risk  of  accidental  generalization  since  statements 
emerging  from  accidental  generalizations  do  not  support  counterfactual  tests  or  modal  imports. 
This  is  a  crucial  distinction  for  laws  have  explanatory  force,  while  accidental  generalizations, 
even  if  they  are  true,  do  not. 

Limitations  to  the  hypotheses  can  also  be  included  by  using  abnormic  law-like  sentences,  i.e., 
law-like  generalizations  that  contains  an  unless-clause.  An  example  of  an  abnormic  law-like 
sentence  is:  The  velocity  of  an  object  does  not  change  unless  the  net  force  on  it  is  not  equal  to 
zero. 

On  the  other  hand,  a  discovery  experiment  focuses  on  identifying  the  aspects  of  interest  of  the 
studied  phenomena.  Its  main  role  is  to  identify  the  essential  elements  that  impact  on  the 
occurrence  of  the  studied  phenomena.  To  ensure  all  possible  elements  are  included,  discovery 
experiments  need  to  be  performed  in  a  realistic  environment.  It  is  therefore  common  to  perform 
discovery  experiments  within  observational  studies  rather  than  within  a  laboratory  setting.  Within 
the  Joint  Fires  Support  (JFS)  Technology  Demonstration  Project  (TDP)  experiments,  the  early 
discovery  technical  integration  experiments  were  performed  using  existing  Canadian  Forces 
Command  and  Control  systems  rather  than  simulated  systems  that  would  have  offered  less 
complexity  but  would  have  not  encapsulated  all  relevant  elements. 

The  exact  number  of  discovery,  hypothesis  testing,  and  demonstration  experiments  within  a  given 
campaign  could  vary  substantially  from  one  campaign  to  another.  The  more  detailed  and  precise 
is  the  tested  concept,  the  fewer  the  number  of  discovery  experiments  required.  The  number  of 
hypothesis  testing  experiments  typically  depends  on  the  breadth  of  the  tested  concepts.  Finally, 
the  number  of  demonstration  experiments  depends  on  the  number  of  stakeholders  and  the 
variation  of  their  interests. 

In  addition  to  designing  the  full  campaign  of  experimentation,  each  individual  experiment  should 
be  defined  carefully  as  several  issues  can  limit  the  validity  of  the  experimental  results.  The  usual 

5  These  properties  should  be  interpreted  carefully.  A  statement  such  as  “Gold  is  malleable”  is  law-like  even 
if  it  refers  to  “Gold”  since  the  statement  is  not  limited  to  any  particular  gold  objects. 
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approach  to  designing  an  experiment  is  described  succinctly  in  Annex  B  and  the  reader  is  referred 
to  the  Guide  for  Understanding  and  Implementing  Defense  Experimentation  (GUIDEX,  [27])  for 
more  details  on  experimentation  best  practices. 

5.2  Experiment  Components 

Every  experiment  can  be  decomposed  into  five  inter-related  components:  Treatment, 
Experimental  Unit,  Effect,  Trial,  and  Analysis. 

•  Treatment.  The  treatment  is  the  causal  factor  to  be  examined  (e.g.,  the  application  of  a  new 
process  or  the  use  of  new  equipment  to  perform  specific  tasks).  It  is  the  object  (abstract  or 
concrete)  that  is  expected  to  influence  the  experimental  result. 

•  Experimental  Unit.  The  experimental  unit  is  the  smallest  unit  assigned  to  the  treatment 
(e.g.,  C-IED  cell,  sensor  operator,  etc.).  It  is  the  system  or  entity  that  performs  the  task 
required  by  the  treatment. 

•  Effect.  The  effect  is  the  dependent  variable  that  relates  to  the  expected  impact  of  the 
treatment  (e.g.,  target  detected  or  not,  time  required  for  accomplishing  the  process,  etc.). 

•  Trial.  A  trial  is  a  single  observation  or  instance  of  the  test  of  the  hypothesis.  The  experiment 
may  consist  of  a  single  trial  or  multiple  trials. 

•  Analysis.  The  analysis  documents  the  outcome(s)  of  the  trials,  compares  the  differing 
conditions  and  treatments  associated  with  those  outcomes,  and  derives  conclusions  and 
recommendations . 

The  typical  components  tend  to  vary  with  the  type  of  experiment.  As  an  example,  the  initial 
discovery  experiments  performed  within  the  JFS  TDP  focused  on  technical  integration.  The 
treatment  was  the  integration  of  systems  and  the  expected  effect  was  a  shared  joint  picture  of  the 
battlespace  with  limited  latency.  Following  JFS  experiments  consisted  in  hypothesis  testing 
experiments  where  the  treatment  was  the  developed  integrated  shared  picture  and  coordination 
tools  and  the  expected  effects  were  improved  shared  situation  awareness  and  a  shorter  response 
time  to  calls  for  fire  support. 

5.3  Experimentation  Benefits  to  CBP 

Experimentation  uses  real  (or  near-real,  i.e.,  laboratory)  environments  to  provide  information  and 
insights  with  a  degree  of  certainty  not  available  from  other  traditional  means  such  as  operational 
research  or  lessons  learned  analysis.  Experimentation  can  provide  evidence  to  verify  whether 
specific  concepts  cause  changes  in  military  effectiveness.  Essentially,  the  benefits  of 
experimentation  include: 

•  Reduced  uncertainty  (risk  mitigation)  with  regards  to  the  employment  of  new  concepts 
(technology  and  methodology); 

•  Objective  evaluation  of  innovations; 

•  Identification  and  possibly  solution  of  practical  problems  that  cannot  be  determined  through 
studies  and  analysis  alone  (validation  of  suggested  issues); 
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•  Provision  of  empirical  evidence  to  inform  doctrine,  policy,  budgetary  and  procurement 
decisions  (i.e.,  supporting  evidence-based  decisions);  and, 

•  Support  to  force  integration  by  enforcing  the  testing  of  new  concepts  and  methodologies 
within  a  joint  and/or  coalition  environment. 

Considering  these  generic  experimentation  benefits  within  the  scope  of  the  CBP  process,  one 
deduces  that  experimentation  could  provide  value  to  the  CBP  process  by  validating  the  mission 
analysis  and  the  prioritization  of  capability  within  the  “Capability  Analysis”  phase,  as  well  as,  by 
providing  empirical  evidence  to  confirm  or  recommended  changes  to  the  IDS  priorities. 
Furthermore,  an  experimentation  campaign  is  not  only  limited  to  the  aspects  considered  within 
the  CBP  process  but  could  also  support  the  testing  of  new  approaches  designed  to  impact  the 
individual-organization  interface.  However,  new  approaches  impacting  on  the  individual- 
organization  interface  might  have  effects  over  a  long  period  of  time  which  could  be  hard  to 
estimate  within  the  current  type  of  experimentation  performed.  Adequate  behavioural 
measurement  tools,  such  as  the  Personal  Value  Questionnaire  [28],  would  be  needed  to  support 
this  type  of  experiments. 

Feedback  from  individuals  who  participated  in  past  Joint  Fires  Support  experiments  indicates  that 
experimentation  actually  plays  a  role  in  improving  the  individual-organization  interface.  In 
particular,  military  operators  indicated  that  through  their  involvement  in  experiments  they  were 
brought  to  better  appreciate  the  overall  targeting  process,  the  constraints  imposed  on  it,  and  the 
potential  avenues  for  improving  this  process.  This  appreciation  from  the  operators  that  actions  are 
taken  to  improve  such  process  will  likely  pave  the  way  for  an  improved  individual-organization 
interface. 


5.4  Experiment  Requirements 

To  optimize  the  benefits  of  experimentation  to  the  overall  force  development  process,  the 
experiment  scenario  must  take  into  account  the  context  of  the  future  security  environment.6  Any 
experimentation  campaign  should  also  leverage  modeling  and  simulation  approaches  since  only  a 
finite  number  of  all  possible  operational  settings  can  be  tested  within  an  experiment.  To  ensure  an 
efficient  use  of  experiments,  one  should  identify  and  focus  on  the  most  relevant  factors  that  will 
impact  the  experimental  results.  This  means  that  planned  experiments  should  leverage  preceding 
experiments  and  available  empirical  data. 

Furthermore,  management  science  and  computer  science  provide  a  wide  variety  of  proven 
conceptual  frameworks  that  can  be  used  to  guide  military  experiments.  As  discussed  in  section  2, 
there  are  various  management  science  frameworks  proposed  by  Lawrence  and  Lorsch,  Thomson, 
Tannenbaum,  Weick,  Barki  and  Pinsonneault  that  could  guide  the  instigation  of  organizational 
integration.  Similarly,  various  frameworks  for  the  integration  of  databases  (Tight  integration, 
Local  As  View,  Global  as  View)  have  been  developed  and  provide  guidelines  for  such  technical 
investigation.  So  far,  only  a  small  amount  of  assessment  tools,  such  as  the  Command  Team 
Effectiveness  (CTEF)  survey  [29],  has  been  developed  using  these  frameworks.  Other  such  tools, 
possibly  leveraging  recent  development  on  Contingency  Theory  [30],  should  be  developed  to 
support  the  assessment  of  management  aspects  beyond  those  included  within  the  CTEF  survey. 


6  Solution  to  today’s  war  might  not  be  ideal  for  tomorrow’s  battles. 
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6  Conclusion 


The  intent  of  this  report  was  to  review  the  existing  force  development  approach  and  compare  the 
current  joint  force  development  approach  with  organization  development  best  practices.  In 
addition,  the  CD&E  activities  were  also  reviewed  focusing  on  the  contribution  of  such  activities 
with  a  broad  organization  development  framework. 

The  review  of  organization  development  science  highlights  the  importance  of  diagnostic  and 
systemic  approach  to  guiding  the  force  development  activities.  Empirical  data  indicates  that  a  too 
narrow  consideration  (i.e.,  incomplete  list  of  requirements)  is  among  the  most  important  causes  of 
project  failure  (see  Table  2).  The  comparative  review  of  joint  force  development  activities 
indicate  two  major  weaknesses  to  the  current  approach:  the  lack  of  leverage  of  the  lessons  learned 
process  to  ensure  a  better  consideration  of  the  problem  space  and  the  lack  of  coverage  of  the 
individual-organization  interface.  These  limitations  to  the  current  process  also  appear  within  the 
CD&E  activities. 

Based  on  the  review  of  organization  development  literature,  the  following  recommendations  are 
made: 

•  The  strategic  and  operational  level  lessons  learned  should  be  adequately  institutionalized  to 
ensure  proper  considerations  of  the  problem  space  within  the  force  development  approach. 
This  input  to  the  force  development  approach  should  be  made  explicit  within  the  overall 
process. 

•  Use  models  that  describe  the  considered  problem- space  should  be  added  to  the  type  of 
concept  models.  These  models  should  build  on  existing  framework  such  as  the  one  proposed 
by  Lawrence  and  Lorsch. 

•  The  force  development  approach  should  be  done  in  a  systemic  fashion  incorporating  all 
aspects  of  the  organization  and  in  particular  the  individual-organization  interface. 

•  Concept  development  should  be  compelling  by  recommending  achievable  targets; 

•  Concept  should  be  precise  enough  for  the  larger  CF  community  to  clearly  be  able  to  identify 
whether  it  is  applied  or  not  when  observing  a  group  of  operators  performing  the  relevant 
operations; 

•  Concept  development  should  be  balanced  by  considering  possible  trade-offs  to  the 
investigated  concept; 

•  Concept  development  should  be  logical  by  building  on  simpler  concepts  in  a  logical  structure; 

•  Experiments  should  be  contextually  relevant  using  scenarios  built  according  to  conclusions 
from  the  future  security  environment;  and, 

•  Additional  assessment  tools  beyond  currently  used  survey  and  data  logs  systems  should  be 
developed  to  support  additional  experimental  aspects.  In  particular,  the  Personal  Value 
Questionnaire  as  well  as  tools  developed  based  on  recent  Contingency  Theory  work  should 
be  considered. 


28 


DRDC  CORA  TM  2012-036 


References 


[1]  Palla,  G.,  Barabasi,  A.L.,  and  Vicsek,  T.  Quantifying  Social  Group  Evolution.  Nature ,  Vol 
446,  pp.  664-667,  5  April  2007. 

[2]  Lawrence,  P.R.  and  Lorsch,  J.W.  Developing  Organizations:  Diagnosis  and  Action.  Addison- 
Wesley  Publishing  Company,  1969. 

[3]  Braunstein,  D.N.  Organizational  Development:  What’s  in  a  Name?  Interfaces ,  Vol.  4,  No.  3, 
May  1974. 

[4]  Schein,  E.H.  Organization  Psychology.  Englewood  Cliffs,  N.J.:  Prentice-Hall,  1965. 

[5]  Pink,  D.H.  Drive:  The  Surprising  Truth  about  What  Motivates  Us.  Riverhead  Books,  2009. 

[6]  White,  R.  Ego  and  Reality  in  Psychoanalytic  Theory.  Psych.  Iss .,  Vol.  Ill,  No.  3,  Monograph 
No.  11,1963. 

[7]  Barki,  H.  and  Pinsonneault,  A.  A  Model  of  Organization  Integration,  Implementation  Effort, 
and  Performance.  Organization  Science  16(2),  2005,  pp.  165-179. 

[8]  Thompson,  J.  D.  Organizations  in  Action:  Social  Science  Bases  of  Administrative  Theory. 
McGraw  Hill,  New  York,  1967. 

[9]  Allen,  D.  Air-Ground  Integration:  Preliminary  Results  from  the  Coalition  Attack  Guidance 
Experiment.  International  C2  Journal.  August  2011. 

[10]  Hull,  E.,  Jackson,  K.,  and  Dick,  J.  Requirements  Engineering.  Springer,  London,  2005. 

[1 1]  Matte,  BGen.  P.R.  Directorate  General  Integrated  Force  Development.  Presentation  to 
CFWC  Orientation  Session,  September  2010. 

[12]  Triantaphyllou,  E.  Multi-Criteria  Decision  Making  Methodologies:  A  Comparative  Study. 
Volume  44  of  Applied  Optimization.  Kluwer  Academic,  Dordrecht,  2000. 

[13]  Burt,  Col.  G.D.  CF  Concepts.  Presentation  at  the  C4ISR  Workshop.  28  September  2010. 

[14]  Chuka,  N.  Re-Examining  the  CF  Joint  Level  Lessons  Identification  and  Analysis  Process  - 
Part  Two:  Applying  the  Learning  and  Innovation  Framework.  DRDC  CORA  LR  2011-046. 

[15]  Carnap,  Rudolf.  Logical  foundations  of probability.  Chicago  University  Press.  1950. 

[  1 6]  Centralized  Command  Decentralized  Control 

[17]  Allen,  D.  An  Overview  of  Factors  Impacting  on  Human  Cognitive  Capacity.  DRDC  CORA. 
Technical  Note.  November  2011.  Draft. 


DRDC  CORA  TM  2012-036 


29 


[18]  NATO  SAS-065.  NATO  NEC  C2  Maturity  Model.  Command  and  Control  Research 
Program.  February  2010. 

[19]  Wolpert,  D.H.  The  Lack  of  A  Priori  Distinctions  between  Learning  Algorithms.  Neural 
Computation ,  1996,  pp.  1341-1390. 

[20]  Wolpert,  D.H.  and  Macready,  W.G.  No  Free  Lunch  Theorems  for  Optimization.  IEEE 
Transactions  on  Evolutionary  Computation.  Vol.  1,  1997,  p.  67. 

[21]  Mukunda,  G.  and  Troy,  W.J.  Caught  in  the  Net:  Lessons  from  the  Financial  Crisis  for  a 
Networked  Future.  Parameters ,  Summer  2009,  pp.  63-76. 

[22]  Wikipedia  contributors,  Formal  versus  Empirical  within  the  “Algorithm”  article. 
Wikipedia,  The  Free  Encyclopedia 

http://en.wikipedia.org/w/index.php?title=Algorithm&oldid=390188121  [accessed  12  Oct 

2010] 

[23]  Allen,  D.,  Comeau,  P.,  and  Farrell,  P.S.E.  Knowledge  Management  as  a  Supporting 
Concept  to  Effects  Based  Operations.  2006  International  Command  and  Control  Research 
and  Technology  Symposium,  Cambridge.  CCRP.  Available  at  www.ccrp.org. 

[24]  Crossley,  J.N.  What  is  Mathematical  Logic?  Oxford  University  Press.  1972. 

[25]  NATO  Allied  Command  Transformation.  Concept  Development  and  Experimentation 
Guide.  Joint  Experimentation,  Exercises  and  Assessment  subdivision,  2009. 

[26]  Hempel,  C.G.  and  Oppenheim,  P.  Studies  in  the  Logic  of  Explanation  in  Philosophy  of 
Science  15,  pp.  135-175,  1948. 

[27]  The  Technical  Cooperation  Program.  Guide  for  Understanding  and  Implementing  Defense 
Experimentation.  Ottawa.  February  2006. 

[28]  England,  G.W.  Managerial  Value  Systems.  In  Emerging  Concepts  in  Management,  Max  S. 
Wortman  and  Fred  Luthans  Eds.  Collier  Macmillan  Ltd,  1969. 

[29]  Essens,  P.,  Vogelaar,  A.,  Mylle,  J.,  Blendell,  C.,  Paris,  C.,  Halpin,  S.  Baranski,  J.  Military 
Command  Team  Effectiveness:  Model  and  Instrument  for  Assessment  and  Improvement. 
NATO  Research  and  Technology  Organization  Human  Factors  and  Medicine  Panel  HFM- 
087/RTG-023,  April  2005. 

[30]  Kalloniatis,  A.,  Macleod,  I.,  Kohn,  E.  Agility  in  an  Extended  Space  of  Constructible 
Organisations.  Presented  at  the  15th  International  Command  and  Control  Research  and 
Technology  Symposium,  June  2010. 


30  DRDC  CORA  TM  2012-036 


Annex  A  Department  of  National  Defence  Architecture 
Framework 


The  following  table  provides  a  brief  explanation  of  the  various  views  and  sub-views  within  the 
DNDAF  Version  1.7.  The  Defence  Entreprise  Architecture  division  is  responsible  for  this  list  of 
views. 


View 

Sub- 

View 

Sub-View  Name 

General  Description 

Common 

CV-1 

Overview  and  Summary 
Information 

Scope,  purpose,  intended  users,  environment 
depicted,  analytical  findings 

Common 

CV-2 

Integrated  Data 
Dictionary 

Architecture  data  repository  with  definitions  of 
key  terms  used  in  sub-views. 

Strategic 

StratV- 

1 

Business  Strategy  and 
Motivation 

Identifies  statements  of  strategic  direction  in 
order  to  guide  and  align  the  business  to  what  it 
would  like  to  become. 

Capability 

CapV- 

1 

Capability  Taxonomy 

Provides  a  structured  list  of  capabilities  activities 
and  sub-activities  that  are  required  within  a 
capability  domain. 

Capability 

CapV- 

2 

Capability  Scenario 
Analysis  Matrix 

Identification  of  relevant  activities,  the  capability 
goals  and  the  mission  effects.  The  capability  and 
mission  effects  assessment  matrix. 

Operational 

OV-1 

High-Level  Operational 
Concept  Graphic 

High-level  graphical/textual  description  of 
operational  concept 

Operational 

OV-2 

Operational  Node 
Connectivity 

Description 

Operational  nodes,  connectivity,  and  information 
exchange  need  lines  between  nodes 

Operational 

OV-3 

Operational  Information 
Exchange  Matrix 

Information  exchanged  between  nodes  and  the 
relevant  attributes  of  that  exchange 

Operational 

OV-4a 

Organization 
Relationships  Chart 

Organization,  role,  or  other  relationships  among 
organizations 

Operational 

OV-4b 

Organization  to 
Role/Skill  Matrix 

Roles  related  to  Organizations  for  an  architecture 
project. 

Operational 

OV-5a 

Functional  Model 

A  chart  that  breaks  down  “what”  the  user  has  to 
do  to  accomplish  his/her  mission. 

Operational 

OV-5b 

Operational  Process 
Model 

A  chart  that  breaks  down  the  procedure  that  is 
followed  among  the  functions  in  order  to 
accomplish  the  mission. 

Operational 

OV-6a 

Operational  Rules 

Model 

One  of  three  sub-views  used  to  describe 
operational  activity — identifies  business  rules 
that  constrain  operation 
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View 

Sub- 

View 

Sub-View  Name 

General  Description 

Operational 

OV-6b 

Operational  State 
Transition  Description 

One  of  three  sub-views  used  to  describe 
operational  activity — identifies  business  process 
responses  to  events 

Operational 

OV-6c 

Operational  Event- 
Trace  Description 

One  of  three  sub-views  used  to  describe 
operational  activity — traces  actions  in  a  scenario 
or  sequence  of  events 

Operational 

OV-7 

Logical  Data  Model 

Documentation  of  the  system  data  requirements 
and  structural  business  process  rules  of  the 
Operational  View 

System 

SV-1 

Systems  Interface 
Description 

Identification  of  systems  nodes,  systems,  and 
system  items  and  their  interconnections,  within 
and  between  nodes 

System 

SV-2 

Systems 

Communications 

Description 

Systems  nodes,  systems,  and  system  items,  and 
their  related  communications  lay-downs 

System 

SV-3 

Systems-Systems 

Matrix 

Relationships  among  systems  in  a  given 
architecture;  can  be  designed  to  show 
relationships  of  interest,  e.g.,  system-type 
interfaces,  planned  vs.  existing  interfaces,  etc. 

System 

SV-4 

Systems  Functionality 
Description 

Functions  performed  by  systems  and  the  system 
data  flows  among  system  functions 

System 

SV-5 

Operational  Activity  to 
Systems  Function 
Traceability  Matrix 

Mapping  of  systems  back  to  capabilities  or  of 
system  functions  back  to  operational  activities 

System 

SV-6 

Systems  Data  Exchange 
Matrix 

Provides  details  of  system  data  elements  being 
exchanged  between  systems  and  the  attributes  of 
that  exchange 

System 

SV-7 

Systems  Performance 
Parameters  Matrix 

Performance  characteristics  of  System  View 
elements  for  the  appropriate  time  frame(s) 

System 

SV-8 

Systems  Evolution 
Description 

Planned  incremental  steps  toward  migrating  a 
suite  of  systems  to  a  more  efficient  suite,  or 
toward  evolving  a  current  system  to  a  future 
implementation 

System 

SV-9 

Systems  Technology 
Forecast 

Emerging  technologies  and  software/hardware 
sub-views  that  are  expected  to  be  available  in  a 
given  set  of  time  frames  and  that  will  affect 
future  development  of  the  architecture 

System 

SV-lOa 

Systems  Rules  Model 

One  of  three  sub-views  used  to  describe  system 
functionality — identifies  constraints  that  are 
imposed  on  systems  functionality  due  to  some 
aspect  of  systems  design  or  implementation 
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View 

Sub- 

View 

Sub-View  Name 

General  Description 

System 

SV-lOb 

Systems  State 

Transition  Description 

One  of  three  sub-views  used  to  describe  system 
functionality — identifies  responses  of  a  system  to 
events 

System 

SV-lOc 

Systems  Event-Trace 
Description 

One  of  three  sub-views  used  to  describe  system 
functionality — identifies  system-specific 
refinements  of  critical  sequences  of  events 
described  in  the  Operational  View. 

System 

SV-11 

Physical  Schema 

Physical  implementation  of  the  Logical  Data 
Model  entities,  e.g.,  message  formats,  file 
structures,  physical  schema. 

Technical 

TV-1 

Standards  Profile 

The  listing  of  standards  that  apply  to  a  system  or 
its  components. 

Technical 

TV-2 

Standards  Forecast 

The  description  of  emerging  standards  and 
potential  impact  on  a  system  or  its  components, 
within  a  set  of  time  frames. 

Information 

IV- 1 

Strategic  Information 
Model 

A  diagram  and  supporting  definitions  that 
describes  the  relationships  between  significant 
high-level  groups  of  data  (information)  and  the 
rules  and  constraints  that  apply. 

Information 

IV-2 

Information 
Accountability  Matrix 

Details  the  accountabilities  of  the  information 
stewards,  and  data  stewards.  It  also  depicts  the 
relationships  between  subject  areas  and  the 
accountability  elements. 

Security 

SecV-1 

Risk  Assessment 

Documents  the  association  of  threats,  risks  and 
the  resulting  security  control  objectives. 

Security 

SecV-2 

Data  Element  Security 
Matrix 

Listing  of  all  data  elements  used  by  the 
architecture  along  with  its  security  parameters. 
Included  in  these  parameters  are  a  means  of 
documenting  the  aggregated  security  implications 
for  each  data  element. 

Security 

SecV-3 

Aggregated  Information 
Security  Matrix 

A  listing  of  all  system  data  exchanges  used  by  the 
architecture  that  may  cause  potential  information 
aggregation  security  violations. 
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Annex  B  Overview  of  Experimentation  Steps 


This  annex  provides  a  brief  overview  of  the  process  used  for  designing  experiments.  The 
experiment  process  described  in  this  annex  described  the  approach  used  at  the  Canadian  Forces 
Warfare  Centre.  This  process  shows  many  similarities  with  the  approach  suggested  within  the 
GUIDEx  [27]  but  is  more  detailed.  Four  consecutives  phases  are  used: 

•  Experiment  framing:  identifying  what  concept  and  methodologies  will  be  tested  and  what  are 
the  experiment  objectives. 

•  Experiment  planning:  Determining  what  needs  to  be  done  and  how  to  proceed. 

•  Experiment  execution:  Developing  all  the  products  required  and  performing  the  experiment. 

•  Experiment  assessment:  Analyzing  the  collected  data  and  producing  reports  and 
presentations. 

Each  of  these  phases  is  illustrated  below.  Green  lines  are  used  to  highlight  the  time  frame  in 
which  each  phase  will  be  accomplished  with  regards  to  the  occurrence  of  the  Concept  Design 
Conference,  the  Initial  Planning  Conference,  the  Mid-Planning  Conference  and  the  Final  Planning 
Conference. 


Overall  Process 


Experiment 
Framing 
(What.  Why) 


Experiment 

Planning 

(How) 


Experiment 

Execution 

(Doing) 


Experiment 
Assessment 
(So  What) 


Problem-Solution 

Scenario 

Experiment  Scenario 

Analysis 

Design 

Development 

Concept  Modeling 

Data  Analysis 

St  Analysis 

St  Runs  Design 

Data  Collection 
&  Analysis 
Set-Up 


Experiment 

Evaluation 

And  Lessons 


Data  Vetting 


Experiment 
Boundary  Framing 


Technical 

Planning 

System 
Development 
St  Testing 

Data  Analysis 
And  Inference 

Experiment 

Integration 

Master  Planning 

Planning 

Training.  Set-Up 
Experiment 
Conduct 


Experiment 

Reporting 
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Experiment  Framing 


Pre-Conference 


Problem-Solution 

Analysis 


Define  Problem 
&  Current 
Solution  (As  Is) 


Define  Identify  Concepts 

Objectives - ►Or  System  (PPT)  Bridging 

(To  Be)  As  Is  &  To  Be 


Concept  Development  Conference 

Define  and 


Concept  Modeling 
&  Analysis 


r 


Experiment 
Boundary  Framing 


I 


Experiment 
Master  Planning 


Model  Concepts  — 
Or  System  CONORS 


Identify 

Concept  Testing - 
Requirements 


Define 

Problem 

Space 


Identify 

Experiment  — 
Objectives  &  Scope 


Generate  Key 
Questions  to  be  — 
Answered  by  Exp. 


Identify 
Assumptions 
&  Constraints 


Exp  Phases 

Planning 

(Timeline) 


Constraints  & 
Resources  _ 

Planning 

(What  unit  should 
Be  involved) 


Develop 
Master  Plan 
(Working  groups, 
R&R,  etc.) 


Experiment  Planning 


^  Initial  Planning  Conference 


Identify  Scenario/Info 
Requirements  and 
Constraints 

Develop  Scenario 


L 


Identify  Relevant 

Variables.  Analysis 
Framework  and  Runs 

i 


* 

Identify  Technical 
Requirements 

Develop  System 


Broad  Line 

1 

Collection  Plan 

1 

1 

l 

Plan  Writing  Boards 

+ 

Develop  Data 

Analysis  Plan 

T 

Develop  System 
Integration  Plan 

^  Mid-Planning  Conference 

| 

Identify  Personnel, 
and  Training 
Requirements 

J 

Develop  Data 

Handling  Plan 

And  Ethics  Protocol 

Develop  Technical 

Security 

Requirements 

Integrated  Planning:  Risk  Assessment  and  Integration  &  Synchronization  Into  Expt  Plan 
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Ex 

periment  Execution 

Experiment 

Data  Collection 

System 

Scenario 

&  Analysis 

Development 

Development 

Set-Up 

&  Testing 

Mid-Planning  Conference  ^ 

Execute  Scenario 
Writing  Board  (MSEL) 
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List  of  acronyms 


AOC 

Air  Operating  Concept 

C4 

Command,  Control,  Communication,  and  Computers 

CBP 

Capability  Based  Planning 

CD&E 

Concept  Development  and  Experimentation 

CDI 

Chief  of  Defence  Intelligence 

CF 

Canadian  Forces 

CFD 

Chief  of  Force  Development 

CFWC 

Canadian  Forces  Warfare  Centre 

CM 

Capability  Management 

CogOC 

Cognitive  Operating  Concept 

COPS 

Collaborative  Operation  Planning  System 

CyOC 

Cyber  Operating  Concept 

DND 

Department  of  National  Defence 

DNDAF 

Department  of  National  Defence  Architecture  Framework 

DR  DC 

Defence  Research  &  Development  Canada 

EC 

Enabling  Concept 

FD 

Force  Development 

FE 

Force  Employment 

FSE 

Future  Security  Environment 

GUIDEx 

Guide  for  Understanding  and  Implementing  Defence  Experimentation 

ICC 

Integrated  Capstone  Concept 

IDS 

Investment,  Divestment  and  Sustainment 

IP 

Investment  Plan 

IT 

Information  Technology 

JCD&E 

Joint  Concept  Development  and  Experimentation 

JIIFC 

Joint  Information  and  Intelligence  Fusion  Capability 

JFS 

Joint  Fires  Support 

KLE 

Key  Leader  Engagement 

LL 

Lessons  Learned 
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LOC 

Land  Operating  Concept 

M&S 

Modeling  and  Simulation 

MOC 

Maritime  Operating  Concept 

OC 

Operating  Concept 

OR 

Operational  Research 

Org 

Organization 

OV 

Operational  Views 

PD 

Personnel  Development 

R&D 

Research  and  Development 

SCR 

Strategic  Capability  Roadmap 

SOC 

Special  forces  Operating  Concept 

SOF 

Special  Operations  Forces 

TDP 

Technology  Demonstration  Program 

TTCP 

The  Technical  Cooperation  Program 
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